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ABSTRACT 


This  thesis  performs  a  comparative  analysis  of  a  sampling  of  risk  management 
plans  for  strategic  and  critical  suppliers  administered  by  the  Defense  Contract 
Management  District  West  (DCMDW)  in  order  to  identify  the  areas  of  highest  risk  and 
the  most  common  tools  used  to  mitigate  risk  in  key  processes  and  systems  for  these 
suppliers. 

The  Defense  Contract  Management  Agency  (DCMA)  uses  a  comprehensive, 
inclusive,  and  iterative  approach  to  risk  management.  It  follows  the  Government  and 
DoD  risk  management  premise  of  using  a  five-step  approach  to  risk  management  and  the 
basic  idea  of  identifying  and  assessing  key  processes/systems  whose  risk,  either  through 
probability  or  potential  impact,  offers  the  most  cause  for  concern  from  a  performance, 
schedule,  or  cost  perspective.  It  employs  current  information  technology,  Risk 
Assessment  and  Management  Program  (RAMP)  to  provide  consistency,  commonality, 
access,  and  comparability  to  its  risk  management  process. 

Performance  and  schedule,  product  support  and  supplier  quality  assurance  for 
product  quality,  and  delivery  were  the  areas  of  highest  risk  for  DCMA.  The  most 
commonly  applied  risk  handling  tools  indicated  in  the  RAMP  database  were  areas 
associated  with  analysis,  monitoring,  and  surveillance  activities  before  final  inspection: 
“Data  Analysis”,  “Product  Audits”,  “System  Evaluation”,  and  “Corrective  Action”. 
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I.  INTRODUCTION 


A.  PURPOSE 

Risk  is  the  probability  of  an  undesirable  event  and  the  significance  of  its 
consequence,  or  more  succinctly,  “an  event  and  its  probability  and  impact  (RM,  1999). 
Within  the  development  of  major  projects,  there  are  five  common  facets  of  risk: 
technical,  programmatic,  supportability,  cost,  and  schedule  and  they  can  be  found  in  all 
phases  of  the  Federal  Acquisition  Process  from  procurement  planning  and  requirements 
analysis  to  the  award  and  post-award  phases.  The  Defense  Systems  Management  College 
(DSMC)  defines  the  risk  management  structure  for  Department  of  Defense  (DoD) 
acquisition  as  a  continuous,  iterative  activity  between  key  processes,  described  as  risk 
planning,  risk  assessment,  risk  analysis,  and  risk  handling.  The  Defense  Contract 
Management  Agency  (DCMA)  is  intricately  involved  in  the  transition  of  major 
acquisition  programs  from  award  to  performance,  and  specifically  in  the  post-award 
contract  administration  phase  of  risk  management.  To  handle  these  responsibilities, 
DCMA  utilizes  risk  management  plans  designed  to  incorporate  all  aspects  of  a  successful 
risk  management  program. 

The  purpose  of  this  thesis  is  to  perform  a  comparative  analysis  of  a  sampling  of 
risk  management  plans  for  strategic  and  critical  suppliers  administered  by  the  Defense 
Contract  Management  District  West  (DCMDW)  in  order  to  identify  the  areas  of  highest 
risk  and  the  most  common  tools  used  to  mitigate  risk  in  key  processes  and  systems  for 
these  suppliers.  Through  a  comprehensive  literature  review,  a  sampling  of  actual  risk 
management  plans,  and  information  gathered  through  interviews  with  DCMA  personnel 
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within  varying  DCMA  resident  and  regional  offices  in  DCMDW,  a  study  of  common 
high  risk  areas  and  risk  mitigation  techniques  is  developed  to  aid  DCMA  personnel  in 
developing  future  risk  management  plans  and  techniques. 

B.  BACKGROUND 

All  acquisition  programs  are  subject  to  risks.  The  DCMA  One  Book  defines  risk 
as  a  measure  of  the  inability  to  achieve  overall  program  objectives  as  defined  by  cost, 
schedule  and  technical  goals.  In  this  context  risk  is  generally  described  by  its  probability 
of  occurrence  and  its  impact.  To  guard  against  risk,  DCMA  has  established  risk 
management  as  an  operating  principle  and  an  integral  part  of  its  processes. 

Risk  management  is  a  systematic  approach  to  problem  solving.  It  includes  risk 
planning,  assessing  risk  areas,  developing  risk  handling  options,  monitoring  risks  to 
determine  how  they  have  changed,  and  documenting  the  overall  risk  management 
program.  Risk  management  plans  are  plans  of  action  to  reduce  or  eliminate  risks 
affecting  cost,  schedule  or  performance.  By  identifying,  analyzing  and  managing  risks 
through  an  iterative,  continuous  program  assessment  via  risk  management  plans,  DCMA 
can  have  significant  positive  impacts  on  the  cost,  schedule,  and  performance  of  its 
assigned  programs. 

Risk  management  plans  identify  and  track  key  risk  drivers,  define  risk  abatement 
plans  and  provide  for  continuous  risk  assessment.  Through  the  use  of  risk  management 
plans  specific  to  each  of  its  suppliers,  DCMA  seeks  to  identify  and  control  critical  risk 
functions  and  bring  them  within  acceptable  levels.  The  Risk  Assessment  and 
Management  Program  (RAMP)  is  an  information  technology  tool  that  readily  allows 
DCMA  to  define  and  document  risk  management  plans  and  to  share  the  information 
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contained  in  these  plans  throughout  the  organization,  and  ultimately  to  its  external 
customers. 

In  RAMP,  DCMA  categorizes  its  suppliers  as  “strategic”,  “critical”,  or  “routine”. 
Strategic  and  critical  suppliers  represent  those  contractors  or  contract  actions  of  highest 
significance  (from  a  cost  or  safety  standpoint)  and  therefore  highest  risk— with  respect  to 
potential  impact: 

•  Strategic  suppliers  are  ACAT I  prime  contractors. 

•  Critical  suppliers  are  those  not  designated  as  “strategic”  and  who 

produce  products/services  classified  as:  ACAT  I  sub-contractor 

(delegation),  safety  of  flight,  flight  critical,  life  support,  explosives, 
munitions,  hazardous,  specialized  safety,  level  1  sub-safe,  nondestructive 
test,  demilitarization,  engaged  in  First  Article  Testing  at  the  time  of  risk 
assessment  (when  RAMP  capable),  space/satellite  (when  RAMP  capable), 
or  nuclear  (when  RAMP  capable). 

•  Routine  suppliers  are  all  those  not  designated  as  “strategic”  or  “critical”. 
(Shields,  2001) 

By  analyzing  the  plans  for  strategic  and  critical  suppliers,  we  can  readily  assess 
how  consistently  DCMA  is  applying  its  own  philosophies,  identify  the  more  significant 
or  high  risk  areas  of  commonality  among  suppliers,  and  recognize  the  more  prevalent  risk 
handling  tools  used  to  mitigate  against  risk  in  the  post-award  contract  administration 
phase  of  acquisition. 

C.  RESEARCH  OBJECTIVE 

The  objective  of  this  thesis  is  to  identify  and  examine  risk  in  the  post-award 
contract  administration  phase  of  the  Federal  Acquisition  Process.  The  goal  of  this  study 
is  to  identify  commonalities  in  the  various  risk  management  plans  of  strategic  and  critical 
suppliers,  determine  the  areas  of  highest  risk  and  define  common  tools  used  to  mitigate 
risk  in  key  processes  and  systems  for  these  suppliers. 
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This  research  serves  as  a  case  study  of  the  risk  management  process  in  DCMDW. 
The  research  will  benefit  DCMA  offices  in  their  continued  implementation  of  the  RAMP 
program  and  its  integration  into  their  current  risk  management  programs  and  processes. 
The  ability  to  see  beyond  the  immediate  supplier  and  recognize  the  commonalities  that 
exist  in  managing  risk  in  the  post-award  contract  administration  phase  of  Government 
acquisition  enhances  the  ability  of  DCMA  personnel  to  make  sound  business  assessments 
and  implement  reasoned  risk  management  approaches  that  have  a  proven  track  record,  a 
higher  probability  of  success,  and  are  consistent  with  Government  and  DoD  risk  handling 
guidance. 

D.  RESEARCH  QUESTIONS 

1.  Primary  Research  Question 

How  does  the  Defense  Contract  Management  Agency  (DCMA)  address  risk 
management  in  the  acquisition  process? 

2.  Subsidiary  Research  Questions 

•  What  is  the  Defense  Contract  Management  Agency  (DCMA)  philosophy 
with  regard  to  risk  management  in  the  post-award  contract  administration 
phase? 

•  Are  risk  management  plans  for  specific  activities  consistently  developed 
and  applied  within  DCMA? 

•  What  are  the  areas  of  highest  risk  for  strategic  and  critical  suppliers  in  the 
contract  administration  phase? 

•  What  are  the  most  common  tools  used  to  mitigate  risk  in  key  processes 
and  systems? 

•  What  is  risk  management  in  the  context  of  the  Federal  Acquisition 
Process? 

E.  SCOPE 

This  thesis  will  be  a  case  study.  The  effort  will  be  directed  to  analyzing  risk 
management  plans  as  documented  in  the  RAMP  database  available  to  DCMDW. 
Interviews  and  opinions  of  key  Government  representatives  involved  in  RAMP 
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implementation  and  the  risk  management  program  will  augment  the  study.  This  research 
will  not  provide  an  exact  template  for  risk  management  plans  rather  it  will  provide  an 
analysis  of  various  risk  management  plans  for  strategic  and  critical  suppliers  assigned  to 
the  DCMDW  and  seek  to  identify  commonalities  that  exist  between  them  to  draw 
conclusions  regarding  areas  of  high  risk  and  risk  handling  tools  in  the  post-award 
contract  administration  phase  of  federal  acquisition. 

Specifically,  this  thesis  will  (1)  review  risk  in  Federal  acquisition  and  specifically 
in  the  post-award  contract  administration  phase;  (2)  present  the  current  DCMA  Risk 
Management  program,  processes,  and  systems;  (3)  analyze  a  representative  sampling  of 
DCMA  risk  management  plans  for  various  strategic  and  critical  suppliers  in  DCMDW; 
(4)  identify  commonalities  in  the  development  and  application  of  Risk  Management 
Plans;  (5)  discuss  areas  of  highest  risk  in  the  contract  administration  phase;  and  (6) 
identify  common  tools  used  to  mitigate  risk  in  key  processes  and  systems. 

F.  METHODOLOGY 

This  thesis  is  a  study  of  risk  management  plans  for  strategic  and  critical  suppliers 

assigned  to  DCMDW.  It  includes  identification  of  commonalities  in  risk  management 

plans,  an  assessment  of  high  risk  areas  common  to  strategic  and  critical  suppliers,  and  a 

presentation  of  common  tools  used  to  mitigate  risk  in  key  processes  and  systems.  A 

comprehensive  literature  review  of  books,  magazine  articles,  CD-ROM  systems,  Internet 

based  materials,  Government  reports,  corporate  materials  and  other  information  sources 

is  conducted  to  describe  risk  in  the  acquisition  environment  and  the  risk  management 

background  within  which  the  DCMA  offices  operate.  The  Defense  Systems  Management 

College  (DSMC)  Risk  Management  Guide  for  DoD  Acquisition  and  the  DCMA  Supplier 

Risk  One  Book  Chapter  are  used  as  guides  for  identifying  risk  areas  and  risk  handling 
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treatments.  A  sampling  of  various  risk  management  plans  for  strategic  and  critical 
suppliers  is  obtained  using  the  RAMP  database  through  the  DCMDW  office.  These  plans 
are  analyzed  to  identify  commonalities  in  their  development  and  application  and 
specifically,  to  look  for  high-risk  areas  and  common  risk  handling  tools  used  to  mitigate 
risk  for  strategic  and  critical  suppliers. 

G.  ORGANIZATION  OF  STUDY 

Following  this  introductory  chapter.  Chapter  II  provides  background  information 
on  risk  management  in  the  Federal  Acquisition  Process.  It  also  provides  an  overview  of 
the  DCMA  Risk  Management  Program  including  its  philosophy  and  various  aspects  of 
the  overall  program  to  include  Process  Oriented  Contracting  Administration  Services 
(PROCAS),  Integrated  Product  Teams  (IPTs),  and  Management  Councils.  Chapter  III 
examines  risk  management  plans  in  the  DCMA  environment.  The  Risk  Assessment  and 
Management  Program  (RAMP)  is  reviewed  including  the  development  and  application  of 
risk  management  plans  throughout  the  agency.  Chapter  IV  presents  data  obtained  from  a 
sampling  of  risk  management  plans  for  strategic  and  critical  suppliers  in  DCMDW  and 
analyzes  the  commonalities  found  within  their  plans,  areas  of  high  risk,  and  risk  handling 
tools.  Chapter  V  includes  the  conclusions  and  recommendation  of  the  thesis.  It  answers 
the  research  questions  and  addresses  topics  for  additional  research. 
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H.  RISK  AND  RISK  MANAGEMENT  BACKGROUND 


A.  RISK  AND  RISK  MANAGEMENT  DEFINED 

Webster’s  defines  risk  as  the  “possibility  of  loss  or  injury”.  (MW,  2001)  Other 

generic  definitions  include  “chance  of  something  going  wrong”  (Encarta,  2001)  or,  when 
used  as  a  verb,  “accept  the  danger  of  doing  it”.  (Cambridge,  2001)  This  seems  simple 
enough,  but  when  asked  “what  is  risk?”  the  term  becomes  hard  to  grapple  with  and,  as 
with  a  lot  of  things,  the  best  answer  may  be  “it  depends”.  Risk  is  a  dependent  concept 
and  typically  one  that  is  thought  of  in  negative  terminology.  One  needs  to  know  the 
context  with  which  it  is  being  used  relative  to  time  and  space  to  properly  define  it  so  that 
it  actually  means  something. 

In  acquisition  related  terms,  DoD  defines  risk  as 

...  a  measure  of  the  potential  inability  to  achieve  overall  program 
objectives  within  defined  cost,  schedule,  and  technical  constraints  ...  [it] 
has  two  components:  (1)  the  probability  (or  likelihood)  of  failing  to 
achieve  a  particular  outcome,  and  (2)  the  consequences  (or  impact)  of 
failing  to  achieve  that  outcome.  (RM,  1999) 

1.  Risk  Characteristics 

Risk  can  be  characterized  in  certain  emotive  and  descriptive  words.  Basic 
characteristics  of  risk  include  volatility,  variance,  uncertainty,  ignorance,  incomplete 
knowledge  and  ambiguity.  Each  of  these  terms  defines  various  aspects  or  dimensions  of 
risk  dependent  upon  the  perspective  from  which  it  is  viewed.  (Shapira,  1995)  Over  time 
and  in  common  use,  risk  has  evolved  from  an  unintended  or  unexpected  outcome  to  an 
outcome  and  a  chance  of  its  occurrence  that  is  decidedly  unfavorable.  (Ansell,  1992) 
Risk,  therefore,  is  commonly  thought  of  in  a  negative  context. 
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Yet  other  methods  for  describing  risk  seem  much  more  scientific — unemotive  in 
nature.  The  George  Washington  University’s  Educational  Services  Institute  (ESI)  course 
on  risk  management  characterizes  risk  as  situational,  time  based,  interdependent, 
magnitude  dependent,  and  value  based  (ESI,  1998).  These  variations  objectively  describe 
and  distinguish  the  nature  and  identity  of  risk  in  the  subjective  context  of  its  environment. 

In  the  context  of  decision-making,  risk  choices  seem  to  embrace  three  dominate 
aspects:  risk  definition,  risk  attitude,  and  risk  management.  How  is  risk  defined  in  the 
situation  at  hand?  What  are  the  decision-makers’  attitudes — risk  adverse,  risk  neutral  or 
risk  seeking?  How  will  the  risk  be  dealt  with?  (Shapira,  1995) 

a.  The  Definition  of  Risk 

Risk  can  be  thought  of  or  described  in  terms  of  its  dimensions  and  its 
relation  to  uncertainty.  Outcomes,  both  positive  and  negative  can  be  considered. 
Parameters  can  be  established  to  consider  how  much  risk  is  used  and  whether 
combinations  of  risk  will  be  regarded  as  descriptive  of  the  whole.  (Shapira,  1995) 

b.  Attitude  Toward  Risk 

The  relationship  between  risk  and  return  will  mold  the  approach  towards 
seeking  or  avoiding  risk  in  varied  situations  dependent  upon  people,  resources,  situations, 
etc..  The  degree  of  risk  and  its  level  of  consequence  will  likewise  impact  the  prevailing 
responses.  (Shapira,  1995) 

c.  Dealing  with  Risk 

The  methods  used  to  handle  or  treat  risk  will  be  dependent  upon  whether 
risks  are  to  be  avoided,  delayed,  or  reduced.  Decisions  include  whether  to  attempt  to 
control  risk,  gather  more  information,  or  merely  change  the  parameters  or  estimates. 
(Shapira,  1995) 
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2.  Risk  Management  Characteristics 

Risk  management  is  “the  act  or  practice  of  dealing  with  risk.  It  includes  planning 
for  risk,  assessing  (identifying  and  analyzing)  risk  areas,  developing  risk  handling 
options,  monitoring  risks  to  determine  how  risks  have  changed,  and  documenting  the 
overall  risk  management  program.”  (RM,  1999) 

The  risks  at  issue  may  be  actual,  but  they  are  also  definitely  what  is  perceived. 
Decisions  are  made  based  on  perceptions  of  risk  consequences.  Perceptions  are  estimates 
of  probabilities  or  likelihood  and  evaluations  of  magnitude  of  outcomes  that  are  often 
subjective;  they  are  psychologically  derived.  Further,  there  is  a  political  dimension 
whereby  decision  makers  are  influenced  by  those  affected  by  the  outcomes.  So  despite 
the  classical  decision  theory  premise  of  objective  calculations,  risk  management  involves 
many  subjective  and  judgmental  contributions.  (Ansell,  1992) 

Given  this,  successful  risk  management  requires  (1)  flexible  and  general  models, 
(2)  a  family  of  related  methods  which  link  models  to  circumstances,  (3)  a  wide  range  of 
skill  and  expertise,  and  (4)  experience  and  leadership.  (Ansell,  1992)  Risky  choice  is 
succinctly  conflict  resolution.  Managers  are  expected  to  manage,  not  just  assess  and 
accept  risk;  they  are  expected  to  “make  things  happen”  and  “take  (good)  risks”.  (Shapira, 
1995) 

Taking  a  less  esoteric  look  and  once  again  seeking  to  more  scientifically  describe 
risk  management,  the  ESI  course  identifies  four  major  components  of  risk  management: 
risk  identification,  risk  quantification,  risk  response  development,  and  risk  response 
control.  (ESI,  1998)  These  concepts  are  developed  further  below. 
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a.  Risk  Identification 

Risks  should  be  identified  early,  often,  regularly,  and  at  all  levels — a 
comprehensive  approach.  The  process  should  be  thorough  and  fully  documented.  Tasks 
should  be  assigned  to  specific  team  members.  Inputs  can  include  such  items  as 
requirements  document,  work  breakdown  structure,  cost  and  time  estimates,  etc.  Tools 
for  idea  generation  include  expert  interviews  and  brainstorming,  etc.  (ESI,  1998) 

b.  Risk  Quantification 

Quantifying  risk  includes  analysis  and  prioritization.  Analysis  includes 
worst,  best,  and  most  probable  scenarios.  Assess  probabilities  and  determine  impacts 
such  as  schedule  risk,  cost  risk,  profitability,  etc.  (Quantifiable  measures  are  preferred, 
but  qualitative  can  be  used.)  Rank  analyzed  risks,  highest  to  lowest  and  filter  out 
unimportant  risks.  (ESI,  1998) 

c.  Risk  Response  Development 

Plan  and  implement  basic  risk  response  strategies  based  on  risk  type. 
Evaluate  and  select  a  primary  option  based  on  a  strategy  of  acceptance  (of  the 
consequences),  avoidance  (eliminate  the  cause),  or  mitigation  (minimize  probability, 
minimize  impact  or  transfer  the  risk).  (ESI,  1998) 

d.  Risk  Response  Control 

Risk  response  control  involves  implementing  risk  strategy,  evaluating  and 
documenting  the  results.  As  risks  become  actual  events,  strategies  are  carried  out. 
Clearly  define  the  lines  of  responsibility,  communicate  status,  and  document  actions. 
Evaluate  the  results  to  reassess  risk  probability,  impact  and  events  as  well  as  risk 
strategies.  Assess  risk  as  to  cost,  schedule  and  performance.  Continually  document  risk 
results:  current,  accurate,  complete,  and  simple.  (ESI,  1998) 
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B.  RISK  MANAGEMENT  IN  THE  FEDERAL  ACQUISITION  PROCESS 

Acquisition  reform  has  changed  the  field  of  play  where  the  Department  of 

Defense  must  buy  its  wares.  Technical,  business  and  management  approaches  have 
evolved.  Today,  it’s  all  about  commercial  products,  streamlined  processes,  and  best 
value.  This  dynamic  arena  is  juxtaposed  against  a  backdrop  of  trimmed  defense  budgets 
and  reduced  Government  oversight  that  makes  for  a  risky  playing  field. 

Risks  can  be  thought  of  in  terms  of  future  events  and  the  uncertainty  associated 
with  their  occurrence  and  potential  impact.  They  are  inherently  interdependent,  time- 
based  and  obviously  situational.  In  acquisition  related  terms,  this  means  risks  exist 
throughout  the  life  of  a  program  and  at  all  phases  of  the  acquisition  cycle.  A  risk 
occurrence  in  one  area  or  phase  will  absolutely  effect  risk  elsewhere,  e.g.,  a  slip  in 
schedule  early  on  in  the  process  will  have  domino  effects  downstream  and  potentially 
impact  other  risk  areas  such  as  cost. 

The  Federal  Acquisition  Process  (FAP)  segregates  acquisition  into  three  distinct 
phases:  pre-solicitation,  solicitation-award,  and  post-award  administration.  Risk  is  alive 
and  well  in  each  of  these  areas.  The  pre-solicitation  phase  includes  risk  associated  with 
such  functions  as  needs  determination,  market  research,  requirements  analysis  and 
sourcing.  The  award  phase  problems  include  issues  such  as  solicitation  method,  selection 
of  contract  type,  bid  or  proposal  evaluation,  and  award  selection.  Transitioning  from 
award  to  contract  administration  involves  risk  associated  with  contract  administration 
plans,  early  DCMA  involvement  (early  PROCAS),  post-award  orientation  conferences 
(the  handoff)  and  flow-down  clauses  for  subcontractors.  (Ross,  1999) 


11 


Risk  management  activities  span  across  all  phases  and  functions  of  the  acquisition 
cycle.  Area  emphasis,  scope  and  detail  will  vary  according  to  phase  and  depend  upon  the 
specific  risk  event.  Though  there  is  no  one  standard  prescribed  for  use,  there  are  some 
general  requirements  and  basic  processes  that  are  common  throughout  the  DoD 
acquisition  arena. 

1.  The  Risk  Management  Process 

The  Department  of  Defense  mandates  the  use  of  risk  management  in  its  major 
defense  acquisition  programs: 

The  acquisition  strategy  shall  address  risk  management.  The  PM 
[Program  Manager]  shall  identify  the  risk  areas  of  the  program  and 
integrate  risk  management  within  overall  program  management.” 

Further,  DoD  encourages  the  use  of  risk  management  throughout  the 
entire  program  life  cycle  and  advocates  “life  cycle  risk  management 
versus  risk  avoidance.  (DoDD  5000.2-R,  2001) 

Given  that  risks  are  to  be  managed  vice  avoided,  DoD  describes  the  overarching 
risk  management  process: 


The  establishment  of  a  risk  management  process  (including  plannings 
assessment  (identification  and  analysis),  handling,  and  monitoring)  to  be 
integrated  and  continuously  applied  throughout  the  program,  including, 
but  not  limited  to,  the  design  process.  The  risk  management  effort  shall 
address  risk  planning,  the  identification  and  analysis  of  potential  sources 
of  risks  including  but  not  limited  to  cost,  performance,  and  schedule  risks 
based  on  the  technology  being  used  and  its  related  design,  manufacturing 
capabilities,  potential  industry  sources,  and  test  and  support  processes;  risk 
handling  strategies;  and  risk  monitoring  approaches....  (DoDD  5000.2-R, 

2001) 

The  Defense  Systems  Management  College  (DSMC)  reiterates  this  basic  risk 
management  process  and  details  its  structure  and  make-up.  This  then  forms  the  basic 
process  model  DoD  prescribes  to  deal  with  acquisition  related  risk. 


12 


Figure  2. 1 .  DSMC  Risk  Management  Structure  (From  RM,  1 999). 

a.  Risk  Planning 

Risk  planning  is  the  process  to  develop  a  risk  management  strategy; 
determine  the  methods  used  to  identify,  analyze,  handle,  monitor,  and  document  risk;  and 
plan  for  adequate  resources  to  implement  the  program.  The  result  is  a  Risk  Management 
Plan  (RMP)  that  is  iterative  and  descriptive  of  the  schedules,  activities,  and  processes. 
The  plan  is  in  essence,  a  road  map.  (RM,  1999) 

b.  Risk  Assessment 

Risk  assessment  is  the  identification  and  analysis  of  risk.  The  process 
begins  with  the  compilation  of  risk  events  and  the  subsequent  evaluation  at  a  level  of 
detail  to  understand  causality  i.e.  risk  drivers  and  impact.  This  problem  identification  is 
the  stage  that  quantifies  the  probability  and  consequences  of  various  risks. 

Risk  assessments  typically  include  a  performance/technical  assessment,  a 
schedule  assessment,  and  a  cost  estimate.  Risk  analysis  activity  begins  with  a  detailed 
study  of  the  critical  risks  to  judge  the  probability  and  impact  on  cost,  schedule,  and 
performance.  Risk  ratings  are  assigned  and  are  often  expressed  as  high,  moderate,  and 
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low  based  on  consideration  of  the  likelihood  of  the  risk  event’s  occurrence  and  its 
consequences.  (RM,  1999) 

However,  it  is  worth  noting  here  that  there  is  no  one  mandated  method  for 
assessing  or  classifying  risk  within  DoD.  For  example,  there  is  no  requirement  to  classify 
risk  as  “high”,  “moderate”,  and  “low”  and  no  specific  method  for  prioritizing  the  risks 
following  the  initial  assessment.  Given  this  predicament,  it  becomes  difficult  to  compare 
risk  or  how  risk  is  handled  between  various  programs  and  activities  or  between  the 
services  themselves. 

c.  Risk  Handling 

Risk  Handling  is  the  specific  methods  and  techniques  used  to  deal  with  the 
identified  risk.  The  chosen  options  are  a  direct  result  of  the  risk  assessment  rating  and 
prioritization.  It  includes  scheduling,  the  assignment  of  responsibility,  and  provides  cost 
estimates.  The  objective  is  to  manage  risks  to  acceptable  levels.  Risk  handling  options 
can  include  risk  avoidance,  control,  transfer  and  assumption.  (RM,  1999) 

(1)  Risk  control  seeks  to  mitigate  risks  to  reduce  the  likelihood 
and/or  consequence  of  their  occurrence.  It  includes  such  activities  as  trade  Studies,  early 
prototyping,  incremental  development,  modeling/simulation,  reviews/inspections,  and 
manufacturing  screening.  (RM,  1999) 

(2)  Risk  avoidance  seeks  to  eliminate  high  or  medium  risk 
sources  and  replace  them  with  a  lower  risk  solution.  This  process  may  involve  such 
efforts  as  changes  in  the  requirements  or  specifications.  An  up-front  requirements 
analysis  and  cost-as-an-independent  variable  (CAIV)  trades  are  sample  risk  handling 
options  here.  (RM,  1999) 
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(3)  Risk  assumption  acknowledges  a  risk  situation  and 
consciously  accepts  the  associated  risk  level  without  especially  seeking  to  exert  any 
control  over  it.  The  basic  premise  here  is  that  not  all  risks  are  worth  worrying  about.  To 
handle  risk  in  this  manner  resources  (time,  money,  people,  administration)  must  be 
identified  to  overcome  the  risk  should  it  occur.  (RM,  1999) 

(4)  Risk  Transfer  reallocates  risk  to  another  part  of  the  system 
during  the  design  phase  or  re-distributes  risk  between  the  Government  and  prime 
contractor  or  between  Government  agencies  or  contracting  team  members.  It  is  a  form  of 
risk  sharing  and  can  influence  cost  objectives  depending  on  where  the  burden  is  placed. 
(RM,  1999) 

d.  Risk  Monitoring 

Risk  monitoring  systematically  tracks  and  evaluates  the  risk  handling 
activities  against  established  metrics.  It  is  a  reiterative  process  which  can  likely  result  in 
changing  and  identifying  new  risks  and  risk  handling  methods.  The  key  measures  here 
are  cost,  schedule  and  performance  effects.  This  is  basically  a  feedback  technique.  Test 
and  Evaluation  (T&E),  demonstration  events,  program  metrics,  and  process  proofing  are 
sample  risk  monitoring  techniques.  (RM,  1999) 

e.  Risk  Documentation 

Formal  documentation  of  the  risk  management  process  offers  several 
benefits.  It  serves  as  a  basis  for  assessments  and  updates,  ensures  a  more  comprehensive 
assessment,  provides  a  method  for  monitoring  and  verifying  results,  provides  background 
material,  is  useful  as  a  management  tool,  and  produces  rationale  for  program  decisions. 
(RM,  1999) 


15 


2.  Post-award  Administration  Phase 

The  Post-award  Administration  Phase  of  Federal  acquisition  (also  known  as 
Contract  Administration)  includes  among  other  functions;  start-up,  quality  assurance, 
payment  and  accounting,  contract  modification,  claims,  termination  and  closeout.  These 
can  be  further  delineated  as  described  in  Figure  1.2.  The  first  four  functions  are  required 
for  every  acquisition,  while  the  last  three  are  dependent  upon  the  administrative 
requirements  specific  to  the  contract  in  question.  Within  this  myriad  of  functions  are 
many  risks  and  many  associated  methods  of  handling  them. 

The  Government  and  contractor  plan  and  initiate  performance  in  the  post-award 
phase.  Large  dollar  contracts  or  contracts  for  complex,  technical  requirements  require  a 
contract  administration  plan  to  delineate  Government  surveillance  and  monitoring 
activities  and  provide  for  proper  Government  and  contractor  performance. 

Various  means  and  methodologies  are  used  to  perform  these  administrative 
functions.  Agencies  may  assign  a  Contracting  Officer’s  Representative  (COR)  or 
Contracting  Officer’s  Technical  Representative  (COTR)  to  liaison  between  the 
Government  and  contractor  to  provide  technical  assistance  to  the  contractor  and  current 
contract  information  to  the  Contracting  Officer  (CO).  The  Defense  Contract 
Management  Agency  (DCMA),  the  principal  organization  for  handling  contract 
administration  within  DoD  acquisition,  assigns  Administrative  Contracting  Officers 
(ACOs)  to  perform  specifically  delineated  contract  administration  functions.  (Ross, 
1999) 
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POST-AWARD  ADMINISTRATION  PHASE 


FUNCTIONS 

SUB-FUNCTIONS 

TASKS 

Start-Up 

Planning 

Ordering 

Subcontracting 

Contract  Administration 
Planning 

Post-Award  Orientations 

Order  Against  Contracts 

Consent  to  Subcontract 

Quality  Assurance 

Monitoring  and  Problem 
Solving 

Property 

Reporting  Performance 
Problems 

Monitor,  Inspect,  and 

Accept 

Delays 

Stop  Work 

Remedies 

Property  Administration 

Report  Performance 

Problems 

Payment  and 
Accounting 

Payment 

Accounting 

Limitation  of  Costs 

Payment 

Unallowable  Costs 

Assignment  of  Claims 

Collecting  Contractor  Debts 

Progress  Payments 

Price  and  Fee  Adjustments 

Accounting  and  Cost 
Estimating  Systems 

Cost  Accounting  Standards 

Defective  Pricing 

Closeout 

Closeout 

Closeout 

Contract  Modification 

Contract  Modification 

Contract  Modification 

Termination 

Termination 

Termination 

Bonds 

Claims 

Claims 

Claims 

Figure  2.2.  FAP  Post- Award  Administration  Phase  (After  Ross,  1999). 

Post-award  contract  risk  management  naturally  follows  all  that  has  gone  before;  it 
builds  on  what  is  already  in  place.  The  process  starts  with  an  Integrated  Baseline  Review 
(IBR)  following  contract  award  to  ensure  plans  and  performance  baselines  are  adequate 
and  consistent  with  the  contract  schedule,  scope,  and  resources.  Although  specific  steps 
to  initiate  the  risk  management  plan  will  vary,  the  following  identifies  some  of  the  more 
basic  ideas: 
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•  Conduct  initial  meeting  with  contractor  to  describe  the  objectives  and 
approach  to  risk  management. 

•  Train  Government  contract  administrators  and  contractors’  organization 
on  risk  management  basics. 

•  Review  pre-contract  award  risk  plan  and  revise  as  necessary. 

•  Conduct  in-depth  review  of  risk  assessments  and  expand  as  necessary. 

•  Review  and  revise  risk  handling  plans  to  match  adjustments  made  in  the 
assessment. 

•  Review  documentation  requirements  with  the  contractor  and  Government 
administration  staff. 

•  Establish  a  formal  risk  management  organization  consistent  with  contract 
terms. 

•  Refine  risk  monitoring  plans  with  the  contractor. 

•  Establish  program  reporting  requirements  with  the  contractor. 

•  Identify  other  risk  management  activities  in  conjunction  with  the 
contractor. 

•  Manage  the  program  risk  in  accordance  with  the  risk  management  plan. 

•  Work  with  contractor  to  refine  risk  monitoring  plans  and  procedures; 
develop  performance  measures  and  metrics  to  track  medium  and  high  risk 
items.  (RM,  1999) 

C.  DCMA  RISK  MANAGEMENT  PROGRAM 

DCMA  is  DoD’s  contract  manager  responsible  for  ensuring  programs,  supplies, 

and  services  are  delivered  on  time,  within  cost,  and  meet  performance  requirements.  As 
of  January  26,  2001,  the  DCMA  homepage  reported  DCMA  manages  325,000  prime 
contracts  valued  at  $852  billion  and  employs  over  12,000  civilian  and  military 
professionals.  Their  employees  interact  with  customers  on  a  daily  basis  to  ensure 
customer  needs  are  met: 

•  Before  Award  -  assist  in  designing  solicitations,  identify  potential 
performance  risks,  select  capable  contactors,  and  write  contracts 
promoting  easy  contract  administration 

•  After  Award  -  ensures  contractor  product,  cost,  and  schedules  are  in 
compliance  with  the  terms  and  conditions  of  the  contract  to  include  on-site 
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surveillance  and  program-specific  processes  that  cannot  be  performed  by 
off-site  buying  activities  (DCMA  web  site,  2001) 

DCMA’s  mission  includes  providing  risk  assessment  services.  DCMA  manages 
risk  using  integrated  supplier  surveillance  planning.  They  preposition  workforce  at 
contractor  sites  and  use  integrated  management  teams  to  bring  a  multifunctional 
perspective  to  assess  supplier  systems  and  processes  for  cost,  schedule,  and 
product/service  performance.  (DCMA  web  site,  2001) 

1.  Philosophy 

The  DCMA  One  Book  (OB)  establishes  risk  management  as  a  central  operating 
principle  and  integral  part  of  DCMA  processes.  DCMA  follows  the  standard  risk 
management  philosophy  prescribed  in  the  DoD  Risk  Management  Guide  (RM)  and  the 
Defense  Acquisition  Deskbook  (DAD).  (OB  0.1,  2001) 

DCMA’s  risk  management  methodology  complies  with  statutory  and  regulatory 
requirements  and  is  specific  to  the  conduct  of  contract  administration  processes.  Agency 
risk  management  is  comprised  of  five  steps:  risk  planning,  risk  assessment,  risk 
handling,  risk  monitoring,  and  risk  documentation.  (OB  0.1, 2001)  Five  mechanisms  are 
used  to  carry  out  the  risk  management  process:  Integrated  Product  Teams  (IPTs); 
contractor  documentation,  product/processes,  metrics  and  data;  Process  Oriented  Contract 
Administration  Services  (PROCAS);  inspection/audit  results  and  data  analysis;  and 
process  mechanisms  for  individual  assessment  tools.  (OB  0.1, 2001) 

2.  Process  Oriented  Contracting  Administration  Services  (PROCAS) 

PROCAS  promotes  mutual  trust  and  understanding  between  the  Government  and 

contractors  by  using  common  objective  data  to  improve  performance  and  encourage 

successful  contract  completion.  PROCAS  adds  value  by  increasing  customer  satisfaction 

through  improved  contract  performance  from  improved  processes  and  increased  on-time 
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delivery.  It  is  a  continuous  process  improvement  approach  that  systematically  provides  a 
method  for  selecting,  analyzing,  and  modifying  processes  so  that  once  a  problematic 
process  has  been  identified  and  corrected,  the  IPT  then  moves  on  to  the  next  process. 
(OB  0.3, 2001) 

All  DCMA  employees  must  use  PROCAS  methods  to  the  maximum  extent 
possible.  Process  improvement  efforts  must  be  conducted  using  Integrated  Product 
Teams  (IPTs)  and  must  be  prioritized  according  to  highest  returns  to  the  Government. 
DCMA  offices  must  maintain  a  history  of  PROCAS  efforts  including  identification  of 
key  processes,  process  analysis,  risk  classifications,  resource  adjustments  and  associated 
cost  savings  or  cost  avoidance.  (OB  0.3,2001) 

DCMA  Contract  Management  Offices  (CMOs)  must  determine  the  degree  of 
involvement  based  on  cost,  schedule,  and  technical  risks  in  the  selected  processes 
affecting  delivery  or  service.  They  must  develop  surveillance  plans  to  form  the  basis  for 
risk  assessment  and  process  identification  and  prioritization.  (OB  0.3,  2001)  However, 
contractors  are  responsible  for  the  processes  employed  to  fulfill  Government  contracts 
and  it  is  their  choice  whether  to  choose  to  team  with  the  Government  to  improve 
processes  of  mutual  interest.  DCMA  members  can  never  mandate  process 
improvements.  (OB  0.3, 2001) 

3.  Integrated  Product  Teams  (IPTs) 

IPTs  are  characterized  as  multi-functional  and  multi-organizational  teams 
designed  to  take  advantage  of  the  disparate  skills  of  their  members.  IPTs  add  value  by 
bringing  various  functional  disciplines  together  to  jointly  build  programs.  Customers  and 
Management  Councils  select  areas  of  potential  benefit  from  IPT  involvement  to  resolve 
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problems,  improve  performance,  facilitate  reform  initiatives,  and  develop  surveillance 
plans.  IPTs  must  use  PROCAS  techniques  and  risk  assessment  tools  to  improve  overall 
contract  performance.  (OB  0.3, 2001) 

IPTs  are  designed  to  promote  cooperation  and  full  and  open  discussions.  Team 
members  should  be  qualified  (in  their  functional  disciplines)  and  empowered  to  speak  for 
their  superiors  or  “principals”  in  the  decision-making  process.  In  fact,  they  are 
encouraged  to  frequently  communicate  with  their  leadership  to  ensure  they  are  espousing 
sound  advice  to  the  sponsoring  party  (Management  Councils,  customers,  etc.)  and  should 
make  other  team  members  aware  of  any  of  their  limitations  to  speak  for  their  parent 
organization.  Agreements  are  considered  “final”  and  therefore  this  continuous  “up-the- 
line”  communication  is  essential.  (OB,  0.3, 2001) 

4.  Management  Councils 

Management  Councils  are  senior  representatives  from  customer  buying  activities, 
program  management  offices,  the  Defense  Contract  Audit  Agency  (DCAA),  DCMA  and 
contractors.  They  are  forums  to  communicate  ideas,  implement  change  and  speed  up 
improvements  in  acquisition.  Here,  all  the  stakeholders  are  brought  together  to 
coordinate  and  resolve  issue  and  thereby  add  value  to  the  acquisition  process.  (OB  0.3, 
2001) 

CMOs  must  establish  and  support  councils  at  all  contractor  sites  that  have  major 
acquisition  programs  (ACAT  I  and  II),  hold  greater  than  80%  of  their  unliquidated 
obligations,  or  on  an  as  needed  basis.  Councils  cannot  alter,  amend  or  deviate  from 
contract  terms  and  FAR/DFARS  requirements.  (OB  0.3, 2001) 
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Contractor  members  must  have  the  authority  to  represent  their  corporation  across 
at  least  two  business  areas,  contractual  entities  or  profit  centers.  All  members  must  be 
senior  enough  to  commit  resources  and  make  decisions  for  their  organizations.  Central  to 
this  is  the  decision  and  the  power  to  establish  IPTs  and  target  specific  processes  for 
increased  risk  management.  The  CMO  Commander,  the  DCAA  Resident  Auditor  and  all 
Program  Managers  and  Item  Managers  must  be  council  members.  (OB  0.3, 2001) 

D.  CHAPTER  SUMMARY 

This  chapter  defines  risk,  provides  some  distinguishing  characters  and  then 
describes  it  in  the  context  of  decision-making.  Risk  management  is  then  introduced 
conceptually  as  a  way  to  deal  with  risk.  It  is  characterized  by  some  basic  over-arching 
functions  that  run  a  common  thread  through  federal  acquisition  and  DoD  risk 
management.  Finally,  the  DCMA  risk  management  program  is  discussed  and  some  of  its 
more  salient  components  presented. 

In  decision-making,  risk  choices  can  be  characterized  in  three  dimensions:  the 
definition  of  risk,  the  attitude  toward  risk,  and  dealing  with  risk.  Risk  management  is  the 
process  used  to  deal  with  risk  and  includes  the  four  broad  steps  of  identification, 
quantification,  response  development,  and  response  control. 

The  Federal  Acquisition  Process  divides  procurement  into  three  phases:  pre¬ 
solicitation,  solicitation-award,  and  post-award  administration.  Varying  risks  and  risk 
treatments  exist  throughout  the  cycle,  but  the  basic  DoD  risk  management  process 
remains  consistent  and  is  mandated  for  use  in  major  defense  acquisition  programs:  risk 
planning,  risk  assessment,  risk  handling,  risk  monitoring,  and  risk  documentation.  It  is  an 
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iterative  process  and  is  carried  through,  continued,  and  expanded  upon  in  the  post-award 
contract  administration  phase  of  the  acquisition  process. 

DCMA  is  the  principal  contract  administrator  for  DoD.  Risk  assessment  services 
are  central  to  their  mission  of  taking  care  of  customer  needs.  DCMA  follows  the 
standard  DoD  risk  management  process  and  uses  several  mechanisms  to  carry  out  this 
function.  IPTs,  PROCAS,  and  the  use  of  Management  Councils  are  central  to  the  risk 
management  philosophy  employed  to  ensure  the  contractor  provides  customer  service 
and  product  delivery. 

The  next  chapter  will  look  at  risk  management  plans  and  how  they  are  used  and 
incorporated  into  DCMA’s  new  risk  management  database,  the  Risk  Assessment  and 
Management  Program  (RAMP).  RAMP  is  designed  to  incorporate  all  aspects  of  the  risk 
management  process  and  provide  a  common  tool  whereby  information  can  be  easily  and 
quickly  shared  with  geographically  dispersed  administrative  offices  and  customers. 
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III.  RISK  MANAGEMENT  PLANS 


A.  INTRODUCTION 

DCMA  has  adopted  a  comprehensive  risk  management  methodology  that  is  to  be 
applied  consistently  to  all  its  suppliers.  Their  Supplier  Risk  Management  Program 
integrates  the  assessment  and  monitoring  processes  and  is  consistent  with  the  stated  DoD 
five-step  process  of  risk  planning,  assessment,  handling,  monitoring,  and  documentation. 
DCMA  has  recently  employed  a  new  tool,  the  Risk  Assessment  &  Management  Program 
(RAMP),  a  computer  software  application,  to  assist  them  in  accomplishing  their  risk 
management  mission. 

RAMP  facilitates  an  integrative  and  iterative  approach  to  the  risk  management 
process.  It  provides  DCMA  with  several  valuable  functions  in  carrying  out  its  contract 
administration  and  subsequent  risk  management  mission: 

•  Provides  one  standard  automated  tool  to  assess  cost,  schedule,  and 
performance  risk. 

•  Facilitates  collection  and  documentation  of  supplier  risk  information. 

•  Requires  supplier  involvement  prior  to  input. 

•  Shares  information  with  buying  activities  (customers)  in  the  form  of  an 
integrated  risk  management  plan. 

In  keeping  with  its  efforts  to  promote  teaming  and  cooperation  with  contractors  as 
encouraged  with  the  PROCAS,  IPT,  and  Management  Council  initiatives,  DCMA 
principles  dictate  that  suppliers  should  be  informed  and  involved  with  risk  management 
actions  and  results.  DCMA  further  stipulates  that  the  generated  output  of  the  RAMP 
system  is  not  to  be  used  by  buying  activities  as  past  performance  data  and  its  use  as 
source  selection  information  is  also  limited.  (OB,  3.1, 2001) 
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B.  SUPPLIER  RISK  MANAGEMENT 

The  DCMA  “One  Book”  chapter  on  Supplier  Risk  Management  establishes  the 

risk  management  policy  and  methodology  DCMA  CMOs  use  to  document  risk 
statements  and  the  required  monitoring  levels  and  techniques  they  use  in  response  to 
specific  contractor  facility  risk.  All  personnel  use  the  planning,  assessment,  handling, 
monitoring  and  documentation  approach  to  perform  these  efforts  at  supplier  facilities. 
Through  risk  management,  DCMA  determines  the  priority,  degree,  and  intensity  of  risk 
handling  and  monitoring  as  well  as  required  resources  needed  at  specific  CMO  locations. 
(OB,  3.1,  2001)  A  detail  of  the  DCMA  risk  management  process  and  responsibilities 
follows. 

Table  3.1.  presents  an  overview  of  the  various  sub-processes  within  the  risk 
management  process  as  a  whole. 

1.  Risk  Planning 

DCMA  Manages  risk  in  the  post-award  contract  administration  phase  through  the 
use  of  CMOs  assigned  to  all  of  its  suppliers.  All  suppliers  and/or  contractual  agreements 
must  have  associated  risk  management  plans.  Functionally  integrated  CMO  teams 
review  the  contract  and  customer  requirements  to  gain  a  clear  understanding  of  the 
customer  needs  and  expectations.  Through  this  procedure  CMOs  identify  key  processes 
and  technical  and  business  systems  that  will  require  surveillance.  Key  processes  are 
identified  by  their  “consequence  of  failure”  on  contract  performance,  schedule,  or  cost. 
(OB,  3.1) 
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SUPPLIER  RISK  MANAGEMENT  PROCESS  OVERVIEW 


Process  Inputs: 

Sub  processes: 

Contract,  Purchase  Order  &  Modifications 

Risk  Planning 

Memorandum  of  Agreement  (MO A),  Quality 

Risk  Assessment 

Assurance  Letter  of  Instruction  (QALI), 

Risk  Handling 

Letter  of  Delegation  (LOD) 

Risk  Monitoring 

FAR  &  DFARS 

Performance  Based  Assessment  Model 

Risk  Documentation 

(PBAM)  Users  Guide 

Process  Mechanisms: 

Performance  Based  Business  Environment 

Functional  personnel  &  IPTs 

(PBBE)  Guides 

Contractor  documentation,  products/ 

Supplier  policies,  procedures,  standards, 

processes,  metrics,  and  data 

and  data 

PROCAS 

Work  Breakdown  Structure  (WBS) 

Supplier  and  program  office  risk 

Inspection/audit  results  &  data  analysis 

Management  plans 

Process  Controls: 

Formal/informal  reviews 

Contractual  terms  &  conditions 

Customer  feedback 

Customer  requirements 

Pre-award  surveys 

CMO  management  review 

DLA-GC  Notifications  of  Suspect  Product 

Unit  Self-Assessment  (USA) 

Internal  Operations  Assessment  (IOA) 
Management  Control  Reviews  (MCRs) 

DLAD  5000.4  Processes: 

Configuration  Mgt  including  Technical  Data 

Performance  Based  Payments 

Contractor  Estimating  System  Reviews 

Progress  Payments  Based  on  Cost 

Contractor  Purchasing  System  Reviews 

Property  Control  System  Analysis 

Contract  Safety 

Public  Vouchers 

Earned  Value  Management  System  (EVMS) 

Schedule  and  Delivery  Management 
Software  Contract  Admin  Services 

Integrated  Logistics  Support 

(SWCAS) 

Material  Mgt  &  Accounting  Systems 

Supplier  Quality  Assurance  (QA) 

Packaging  Management  Program 

System  Planning,  RD&E 

Parts  Control  Program 

Test  and  Evaluation  Management 

Table  3.1 .  Supplier  Risk  Management  Process  (After  OB,  3.1, 2001). 

2.  Risk  Assessment 

DCMA  must  perform  a  risk  assessment  for  all  its  suppliers.  Performance, 

schedule,  and  cost  are  the  principal  areas  of  consideration.  The  CMO  team  or  functional 

specialist  will  assign  a  risk  rating  to  each  system  or  key  process  based  on  a  combination 
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of  high,  moderate,  and  low  ratings  for  likelihood  of  failure  and  impact.  The  key 
indicators  of  risk  are  a  contractor’s  experience,  performance,  and  capability.  The  rater 
must  be  able  to  support  the  assigned  risk  ratings  with  actual  data  representative  of  these 
key  indicators:  proofing,  audits,  evaluations,  etc;  both  contractor  and  Government 
supplied  information  can  be  used  for  this  purpose.  For  assistance,  the  rater  can  research 
each  applicable  One  Book  process  and  its  associated  Risk  Matrix  to  ascertain  specific 
performance  requirements  that  relate  to  the  contractor  in  question.  (OB,  3.1, 2001) 

Figure  3.1  depicts  the  risk  assignment  process  using  a  matrix  table  to  produce  risk 

ratings. 


The  following  risk  ratings  are  appropriate  under  the  listed  conditions.  Key 
definitional  differences  are  denoted  by  the  italicized  words. 
a.  High  Risk 

•  Failure  or  nonconformance  likely  to  result  in  unsafe  conditions  for 
personnel. 

•  Failure  of  nonconformance  likely  to  result  in  mission  failure  or  prevent 
proper  tactical  function  of  a  major  end  item  (aircraft,  weapon,  or  space 
system) 

•  Process  is  out  of  control. 

•  Performance  data  indicates  significant  doubt  of  system  or  process 
capability  to  meet  requirements. 

•  A  major  disruption  is  highly  probable  and  the  contractor  is  unlikely  to 
meet  performance,  schedule  or  cost  objectives. 
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b.  Moderate  Risk 

•  Failures  could  result  in  unsafe  conditions. 

•  Failures  could  adversely  affect  mission  performance. 

•  Proper  performance  of  end  items,  subassemblies,  or  key  processes  is 
doubtful. 

•  There  is  a  moderate  process  variance  and  the  trend  is  adverse. 

•  Performance  data  indicates  doubt  of  system  or  process  capability  to 
consistently  meet  requirements. 

•  Probable  that  the  contractor  will  encounter  delays  and  if  concerns  are  not 
addressed  the  process  may  progress  to  “high”  risk. 

c.  Low  Risk 

•  Failures  are  unlikely  to  present  serious  problems. 

•  Performance  data  provides  confidence  in  system  or  process  capability  to 
meet  requirements. 

•  Minimal  or  no  impact  in  meeting  performance,  schedule,  or  cost 
objectives.  (OB,  3.1, 2001) 
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3.  Risk  Handling 

DCMA  teams  or  specialists  must  use  risk  handling  plans  as  the  operational  risk 
management  tool.  The  plans  must  specify  the  methods  used  to  mitigate  risk  associated 
with  a  contractor’s  systems  or  key  process.  CMOs  may  use  either  Government  action  or 
Contractor  Self-Oversight  as  the  surveillance  method. 

IPTs  or  functional  specialists  develop  and  execute  risk  handling  plans  as  required 
according  to  applicable  DLAD  5000.4  One  Book  policy  chapters  or  as  indicated  by  other 
technical  and  business  systems  presenting  risk.  The  risk  handling  plan  indicates  the 
intensity,  schedule  and  frequency  of  the  designated  risk  handling  method.  CMO 
personnel  can  apply  PROCAS  methods  (process  proofing,  product  audits,  data  analysis, 
etc.)  to  any  of  the  risk  areas  to  improve  systems,  processes,  or  products.  (OB,  3.1,  2001) 
Some  examples  of  risk  handling  methodologies,  given  risk  specific  situations  follow. 

a.  High  Risk 

•  Immediate  and  intensive  surveillance. 

•  Establish  intensive  system  evaluations,  product  audits,  process  proofing, 
data  analysis,  root  cause  analysis,  corrective  action,  and  statistical 
sampling. 

•  Execute  until  risk  is  mitigated  to  a  lower  level. 

b.  Moderate  Risk 

•  Intensity  and  frequency  of  surveillance  includes  establishment  of 
scheduled  system  evaluations,  product  audits,  process  proofing,  data 
analysis,  root  cause  analysis,  corrective  action,  and  statistical  sampling. 

•  Execute  until  risk  of  impact  is  reduced. 

c.  Low  Risk 

•  Intensity  and  frequency  of  surveillance  includes  using  periodic 
Government  and  contractor  data  reviews  (EVMS,  delivery  performance 
history,  process  control  data,  cost  control  data,  extensive  audit  data,  etc.) 

•  Ensure  process  variance  does  not  increase  and  process  capability  remains 
stable.  (OB,  3.1, 2001) 
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CMOs  must  have  a  specific  risk  handling  plan  for  all  suppliers  at  all  given 
locations  regardless  of  complexity,  risk  level,  or  dollar  value  of  contract(s).  The  plans 
must  be  tailored  to  the  program,  contract,  or  supplier  facility.  The  depth  and  length  of  the 
plans  vary  and  depend  upon  business  volume,  product  criticality,  or  acquisition 
complexity.  The  plans  may  be  contract  specific  (when  the  requirement  is  not  applicable 
to  all  contracts  within  a  facility)  or  facility  specific  (when  the  process  or  system  is 
common  to  all  contracts  within  the  facility).  (OB,  3.1, 2001) 

4.  Risk  Monitoring 

The  DCMA  team  or  specialist  must  track  and  evaluate  performance  relating  to 
systems  and  key  processes  identified  in  the  risk  handling  plan.  Monitoring  involves 
constant  and  consistent  follow-up  of  all  that  has  gone  before  through  the  regular  use  of 
surveillance  methods  that  will  truly  measure  contract  performance.  Assigned  personnel 
will  compare  results  with  objectives  for  the  various  risk  handling  methods  and  adjust  the 
methods,  intensity,  and  frequency  accordingly.  This  is  basically  trend  analysis,  an 
important  indicator  of  future  success.  Adverse  results  may  require  the  IPT  or  specialist  to 
take  corrective  measures  and  increase  surveillance.  They  will  modify  risk  assessments 
and  the  risk  handling  plans  as  needed  to  account  for  the  results  of  the  ongoing  risk 
management  program.  (OB,  3.1, 2001) 

5.  Risk  Documentation 

The  team  or  specialist  must  record  and  maintain  current  documentation  of  the 
entire  risk  management  program  and  any  updates  as  required.  (OB,  3.1, 2001) 

Figure  3.2  displays  the  Supplier  Risk  Management  Process  as  a  flow  chart,  clearly 
indicating  the  iterative  nature  of  the  risk  management  process. 
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SUPPLIER  RISK  MANAGEMENT  PROCESS  FLOW 


Figure  3 .2.  Supplier  Risk  Management  (After  OB,  3.1, 200 1 ) 

C.  RISK  ASSESSMENT  AND  MANAGEMENT  PROGRAM  (RAMP) 

1.  Transition 

RAMP  software  is  the  mandated  tool  for  risk  assessment  and  handling  activities 


throughout  DCMA.  It  integrates  and  automates  these  processes  and  eases  collection  and 


documentation  of  supplier  risk  information.  RAMP  is  consistent  with  DCMA 
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Information  Technology  policy  for  mission  applications  and  replaces  all  local  automated 
risk  assessment  tools  that  were  previously  in  use.  RAMP  is  a  module  in  the  web-based 
Supplier  Information  Service  (SIS)  and  is  open  to  DCMA  customers,  basically  supplying 
the  same  information  previously  shared  through  other  channels:  IPTs,  Management 
Councils,  etc.  (IM  00-223, 2000) 

By  implementing  RAMP,  DCMA  CMO  personnel  have  transformed  from 
conducting  periodic  risk  assessments  using  a  Performance  Based  Assessment  Model 
(PBAM)  that  required  only  tri-annual  full-up  evaluations,  with  annual  desk  audits 
dispersed  in  between,  to  real-time  supplier  surveillance.  The  new  process  integrates  the 
PBAM  risk  assessment  and  surveillance  planning  processes  to  institute  a  consistent  risk 
management  methodology  throughout  DCMA.  (TM  99-79,  1999)  When  implementing 
RAMP,  CMO  personnel  are  able  to  initially  populate  the  RAMP  database  with  previously 
used  PBAM  information  due  to  the  fact  that  the  tenets  of  the  program  remain  consistent 
with  the  new  policies.  (IM  00-223, 2000) 

A  new  Supplier  Risk  Management  One  Book  Chapter  was  added  to  be  the 
“overarching”  policy  for  the  new  risk  management  program.  Additionally,  20  One  Book 
Chapters  “link  to”  and  supplement  the  guidance.  Process  Owners  were  required  to 
update  policy,  guidebooks  and  training  strategy  to  accommodate  these  changes.  The  Risk 
Assessment  and  Management  Program  (RAMP)  is  the  automated  tool  associated  with  the 
supplier  risk  management  program.  (IM  99-273,  1999) 

The  RAMP  database  will  remain  closed  to  DCMA  suppliers  due  to  the  fact  that  it 
is  intended  solely  to  be  a  DCMA  internal  management  tool  designed  to  automate  policy. 
However,  DCMA  operating  principles  encourage  a  teaming  approach  with  its  suppliers, 
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as  evidenced  through  other  acquisition  reform  initiatives:  PROCAS,  IPTs,  Management 
Councils,  etc.  With  this  in  mind,  supplier  information  is  to  be  shared  and  discussed  with 
the  cognizant  contractors  prior  to  use  within  the  RAMP  system 

2.  Organization 

A  RAMP  risk  management  plan  is  organized  to  assign  risk  ratings  at  four 
different  levels:  Overall,  Service  Set,  One  Book  chapter,  and  key  process/system.  Five 
service  sets  support  the  overall  rating  and  20  One  Book  Chapters  and  their  associated  key 
processes/systems  define  the  service  set.  CMO  personnel  will  assign  three  separate 
ratings  for  performance,  schedule,  and  cost  to  identified  risks  at  each  of  these  levels.  If 
no  risk  is  identified,  then  the  area  will  remain  un-rated.  However,  an  “overall”  rating 
must  be  assigned  to  the  supplier  or  contract  as  a  whole.  (IM  00-293, 2000) 


The  following  table  displays  the  five  service  sets  employed  in  the  RAMP  database 


and  the  associated  20  One  Book  Chapters  used  to  evaluate  contractor  risk: 


ONE  BOOK  POLICY  STRUCTURE 

SERVICE  SET  ALIGNMENT 

Major  Program 

Earned  Value  Management  System  (EVMS) 
Acquisition  Logistics  Support 

Delivery 

Schedule  and  Delivery  Management 
Contract  Safety 

_ 

Business  and  Financial  Systems 

Contract  Property  Management 

Contractor  Estimating  System  Reviews 
Contractor  Purchasing  System  Reviews 

Material  Management  &  Accounting  Systems 

Product  Support 

System  Planning  RD&E  -  Design  Eng 
SPRD&E  -  Systems  Eng 

Test  &  Evaluation  Management 

Supplier  QA  -  Quality  System 
Configuration  &  Technical  Data  Mgt 
Packaging  Management  Program 

Parts  Control  Program 

Software  CAS 

Supplier  QA  -  Product  Quality 

Payment  &  Financial  Mgt 

Progress  Payments  Based  on  Cost 

Performance  Based  Payments 

Public  Vouchers 

_ 

Table  3.2.  Service  Set  Alignment  (After  SRM  Brief,  2000). 
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3.  Responsibilities 

Functional  specialists  populate  the  initial  database  and  enter  data  into  RAMP. 
They  rate  their  processes  and  systems  and  overall  One  Book  Chapters  in  performance, 
schedule,  and  cost.  This  rating  is  a  professional  judgment  call  that  should  take  into 
account  supporting  and  verifiable  information.  This  is  justified  by  a  written  narrative  that 
describes  the  information  used  to  support  the  ratings.  (IM  00-293, 2000) 

Service  Set  ratings  are  system  generated  from  their  supporting  One  Book  ratings 
and  cannot  be  altered.  CMO  designated  Supervisor/Team  Leaders  will  review  the  ratings 
at  this  level  and  provide  a  written  narrative  that  summarizes  the  assessment  and 
prescribed  risk  handling  activities.  (IM  00-293, 2000) 

The  CMO  designated  Operations  Group  Leader  or  Team  Leader(s)  will  review  all 
RAMP  information  and  assign  an  overall  rating  to  performance,  schedule,  and  cost  and 
write  a  supporting  narrative.  These  ratings  are  system  generated,  but  can  be  changed  at 
the  discretion  of  CMO  management.  (IM  00-293, 2000) 

4.  Risk  Rating  Assignments 

A  risk  rating  of  “high”,  “moderate”,  or  “low”  is  assigned  to  performance, 
schedule,  and  cost  for  the  supplier  or  contract  overall  and  for  the  identified  risk  areas  at 
each  of  the  other  four  levels.  Some  service  sets  or  One  Book  chapters  may  receive  no 
rating  at  all:  “NA”  or  “not  applicable”. 

If  there  are  no  historical  contractor  data,  second  party  data,  or  working  records  for 
a  new  contractor,  the  key  processes  or  systems  identified  for  risk  assessment  should  be 
considered  “in  process”  for  performance,  schedule  and  cost  until  data  can  be  reviewed. 
This  is  a  temporary  rating  until  a  functional  specialist  can  review  first  output.  There 
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should  however,  be  rationale  (narrative)  for  this  area  since  it  was  chosen  as  a  priority  for 
risk  assessment  in  the  first  place.  (IM  00-293, 2000) 

In  and  of  themselves,  areas  that  buyers  require  DCMA  to  monitor  are  not 
automatically  considered  “high”  risk.  The  risk  rating  assigned  by  the  CMO  is  a 
combination  of  likelihood/probability  and  impact  should  the  risk  event  occur.  Customer 
specified  “important”  characteristics  are  a  contributing  factor  when  specialists  rate  the 
impact  or  consequence  side  of  the  risk  matrix.  (IM  00-293, 2000) 

All  delegated  subcontract  work  shall  be  entered  into  RAMP  to  provide  customer 
visibility  of  this  level.  If  the  contractor’s  delegation  only  specifies  product 
characteristics,  DCMA  personnel  should  identify  the  actual  subcontractor  processes)  that 
produce  these  specified  product  characteristics.  (IM  00-293,  2000) 

5.  Supplier  Risk  Handling 

DCMA  risk  assessments  and  the  resulting  risk  ratings  are  designed  to  be  based  on 
verifiable  and  producible  data  that  contractors  can  review.  Government  judgment  calls 
alone  are  usually  not  enough  to  convince  a  contractor  to  take  additional  measures  to 
guard  against  potential  error.  The  data  used  to  support  the  risk  rating  may  be 
Government  or  contractor  collected,  so  long  as  they  produce  clear  evidence.  (IM  01-020, 
2001)  Although  the  Government  may  plan  and  assess  risk,  it  is  the  supplier  who  must 
actually  handle  it  to  change  the  way  a  process  works. 

As  part  of  its  assignment  in  facilitating  DoD’s  risk  management  program,  DCMA 
has  a  role  to  influence  the  risk  handling  the  contractor  may  voluntarily  perform.  This  is 
where  the  “narrative”  aspect  of  RAMP  assessment  often  proves  to  be  helpful  by 
providing  a  clear  cause-and-effect  trail  for  the  contractor  to  consider.  A  “cooperative” 
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approach  similar  to  a  PROCAS  agreement  is  the  preferred  course  of  action  within 
DCMA.  Failing  this,  a  Corrective  Action  Request  (CAR)  is  an  alternative  tool  that  may 
need  to  be  utilized  by  the  CMO.  (IM  01-020, 2001) 

Once  risk  is  identified,  suppliers  may  choose  one  of  four  risk  handling  options  to 
deal  with  a  risky  process  or  system: 

•  Avoid  risk  by  changing  the  situation  so  that  risk  is  no  longer  present  i.e. 
restructuring. 

•  Accept  risk  by  acknowledging  its  likelihood  and  consequences  and 
(hopefully)  plan  for  its  contingency  if  it  occurs. 

•  Transfer  risk  to  another  system  or  location  where  the  impact  is 
minimized. 

•  Control  risk  by  reducing  the  likelihood  (prevention)  or  the  impact 
(reduction).  (IM  01-020, 2001) 

6.  Government  Monitoring 

This  is  actually  risk  handling  performed  by  the  Government.  Since  the 
Government  cannot  actually  alter  the  process  or  system — having  no  such  ownership  over 
these  areas — DCMA  must  conduct  continuous  data  review  to  “pulse”  identified  key 
processes/systems  considered  risky  to  the  contractor’s  overall  performance.  This 
basically  involves  gauging  the  movement  of  measured  outputs  (or  trend  analysis)  from 
the  risk  handling  tools  chosen  to  mitigate  the  risk. 

Intensity,  frequency  and  schedule  are  used  to  describe  the  risk  handling  tools  for 
each  key  process/system  identified  as  requiring  risk  management.  “Intensity”  measures 
the  degree  to  which  the  specific  tool  is  to  be  applied,  e.g.,  100%,  sample  size,  specific 
elements.  (IM  00-293,  2000)  “Frequency”  describes  the  periodicity  of  the  risk  handling 
action  and  “schedule”  provides  a  more  specific  time  reference. 
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This  area  of  the  risk  management  process  also  includes  any  “mandatory 
inspections”  customers  require.  From  the  DCMA  perspective,  an  inspection  is  only  as 
good  as  its  output’s  relevancy  to  managing  risk:  "...  risk  is  not  reduced  unless  a  process 
is  changed  and  inspection  does  not  change  a  process.”  (IM  01-020,  2001)  Inspection  is 
the  monitoring  of  performance  and  assists  in  determining  whether  the  risk  handling 
methodology  needs  to  be  changed  to  improve  performance. 

7.  Risk  Management 

RAMP  produces  what  is  more  accurately  considered  to  be  Risk  Management 
Plans  because  they  encompass  all  five  aspects  of  the  risk  management  process  (planning, 
assessment,  handling,  monitoring,  and  documentation)  and  not  just  the  “risk  handling” 
requirements  called  for  under  DCMA’s  supplier  risk  management  policy.  (IM  01-020, 
2000) 

In  maintaining  its  status  as  a  real-time  risk  management  tool,  RAMP  will  be 
updated  as  needed  to  report  current  conditions  at  supplier  locations.  As  prescribed  by 
DCMA  policy,  the  maximum  frequency  between  updates  is  one  year.  Personnel 
responsible  for  updating  the  database  (the  functional  specialists)  are  tasked  with  keeping 
abreast  of  changing  conditions  at  contractor  sites  that  could  result  in  changing  risk  ratings 
and  priorities,  e.g.,  reorganizations,  strikes,  renovations.  (IM  00-293, 2000) 

RAMP  is  intended  to  facilitate  the  collection  of  supplier  information  for  the 
purpose  of  contract  management.  It  provides  the  framework  for  a  systemic  approach  to 
assigning  risk  ratings  that  are  used  by  DCMA  personnel  to  identify  and  prioritize  process 
improvements  as  well  as  resource  allocation.  However,  despite  this  substantial  gathering 
of  performance  information  on  specific  contractors,  DCMA  has  established  policy  that 
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expressly  prohibits  the  use  of  RAMP  data  for  pre-award  source  selection  past 
performance  information.  “RAMP  is  intended  to  be  used  as  a  post-award  system,  not  as 
a  past  performance  tool  because  it  does  not  have  the  appropriate  checks  and  balances 
necessary  for  that  purpose.”  (IM  01-115, 2001) 

D.  CHAPTER  SUMMARY 

This  chapter  begins  with  a  discussion  of  DCMA’s  supplier  risk  management 
process  including  the  risk  analysis  matrix  used  for  assigning  risk  ratings  and  the 
associated  risk  handling  methodologies  applied  to  the  various  rating  levels.  It  proceeds 
with  DCMA’s  incorporation  of  current  risk  management  information  into  the  new 
automated  risk  management  program,  RAMP.  Finally,  various  aspects  of  the  RAMP 
database  are  reviewed  and  pertinent  program  application  issues  detailed. 

DCMA  has  adopted  a  comprehensive  risk  management  methodology  to 
consistently  apply  to  all  its  suppliers.  DCMA  created  a  new  One  Book  Chapter  to 
describe  its  risk  management  process  and  assign  responsibilities  to  its  CMOs.  It  employs 
a  risk  matrix  structure  to  define  risk  in  terms  of  probability  and  consequence  and  assign 
risk  ratings  for  performance,  schedule,  and  cost.  Risk  handling  methodologies  vary  in 
intensity  as  appropriate  to  mitigate  the  associated  level  of  risk. 

The  RAMP  program  is  designed  to  be  an  all  encompassing  risk  management  plan 
incorporating  all  five  functions  of  the  risk  management  process  in  one  automated  tool 
allowing  users  from  geographically  dispersed  sites  to  share  data.  The  RAMP  database 
will  be  initially  populated  with  existing  risk  management  plans.  Its  information  will  be 
made  available  to  customers  although  it  is  expressly  not  to  be  used  for  past  performance 
data  or  source  selection  criteria.  RAMP  will  remain  closed  to  suppliers;  however,  in 
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keeping  with  the  tenets  of  PROCAS,  IPTs,  and  Management  Councils,  the  information 
will  be  shared  and  discussed  with  contractors  prior  to  use. 

RAMP  is  the  mandated  tool  for  risk  assessment  and  handling  throughout  DCMA. 
DCMA  functional  specialists  or  IPTs  will  identify  risk  priorities  and  assign  risk  ratings  at 
the  key  process/system  and  One  Book  Chapter  level;  management/supervisory  oversight 
will  review  automatically  generated  ratings  at  the  Service  Set  and  Overall  rating  levels 
and  write  narrative  cause-and-effect  descriptions  to  support  the  assigned  risk.  However, 
the  role  of  risk  handling  belongs  to  the  contractor;  it  is  the  supplier’s  process  that  must  be 
adjusted  and  only  the  contractor  can  do  this.  Hence,  the  ongoing  teaming  aspects  of 
DCMA’s  risk  management  program. 

The  next  chapter  will  present  risk  management  data  from  a  sampling  of  risk 
management  plans  representative  of  the  Defense  Contract  Management  District  West 
(DCMDW)  region.  A  comparative  analysis  of  these  plans,  obtained  from  the  RAMP 
database,  will  be  conducted  to  identify  commonalities,  high  risk  areas,  and  risk  handling 
tools  consistent  across  the  region. 
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IV.  RAMP  DATA  PRESENTATION  AND  ANALYSIS 


A.  INTRODUCTION 

The  purpose  of  this  chapter  is  to  present  and  analyze  risk  management  data 
obtained  from  a  sampling  of  risk  management  plans  from  the  RAMP  program  initiated  in 
DCMDW.  Forty-two  (42)  RAMP  plans  from  strategic  and  critical  suppliers  are 
reviewed.  The  analysis  focuses  on  commonalities  between  the  plans  themselves  and  the 
requirements  as  set  forth  by  DCMA  and  DoD.  It  studies  areas  of  highest  risk  in 
performance,  schedule,  and  cost  for  the  suppliers  overall  and  at  the  service  set  and  One 
Book  chapter  levels  of  review.  Further,  it  researches  common  risk  handlings  tools 
selected  to  deal  with  the  various  risks  identified  at  the  key  process/system  level  of 
planning. 

DCMDW  manages  more  that  125,000  contracts  totaling  over  $500  billion.  The 
district  consists  of  15  field  offices  on-site  at  contractor  facilities,  13  geographic  offices 
handling  multiple  suppliers  for  specified  areas  within  the  region,  and  a  headquarters 
office  in  Carson,  CA.  (DCMDW  web  site,  2001)  As  of  April  30,  2001  DCMDW’s 
RAMP  database  population  includes  117  strategic  assessments  and  857  critical  plans 
from  5,375  total  assessments  for  the  entire  region.  The  strategic  and  critical  assessments 
come  from  only  27  strategic  and  718  critical  suppliers  respectively.  (Shields,  2001) 
Many  suppliers  have  more  than  one  plan  due  to  multiple  contracts. 

The  sample  analyzed  here  includes  42  plans  from  eight  different  geographic  and 
in-plant  offices  in  DCMDW.  The  plans  represent  30  different  contractor  organizations 
and  a  cross  section  of  facility-wide  and  contract(s)  specific  risk  management  plans.  The 
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plans  are  all  from  critical  and  strategic  suppliers.  Appendix  A  provides  a  listing  of  the  42 
sampled  RAMP  risk  management  plans  and  their  associated  offices,  locations,  and 
suppliers. 

B.  OVERALL  RISK  RATINGS  &  SERVICE  SET  SUMMARY 

Appendix  B  presents  an  overview  of  the  Overall,  Service  Set,  and  One  Book 

Chapter  risk  ratings — high  (H),  moderate  (M  or  Mod),  low  (L) — in  performance  (P), 
schedule  (S),  and  cost  (C)  for  each  of  the  42  RAMP  plans.  Plans  without  final  ratings  are 
indicated  as  in  process  (IP). 

Table  4.1  provides  an  overview  of  the  overall  risk  ratings  for  performance, 
schedule,  and  cost  for  the  42  sampled  plans  from  critical  and  strategic  suppliers.  One 
plan  listed  overall  risk  ratings  as  “in  process”  and  is  consequently  not  included  in  the 
tabulation. 


OVERALL  RISK  RATINGS 

High 

Mod 

Low 

Performance 

7 

17 

17 

Schedule 

6 

17 

18 

Cost 

5 

8 

28 

Table  4.1.  Overall  Risk  Ratings. 
(Source:  Developed  by  Researcher.) 


Overall  risk  ratings  are  system  generated,  but  can  be  changed  at  the  discretion  of 

the  rater;  however,  the  method  employed  on  each  individual  plan  is  not  readily 

discemable  by  the  reader.  Often  the  risk  ratings  appear  to  be  average  assessments  based 

upon  ratings  achieved  at  the  Service  Set  level,  which  are  in  turn  driven  by  One  Book 

Chapter  risk  ratings.  However,  there  are  instances  where  a  high  risk  rating  overrides 

what  would  otherwise  be  a  lower  rating  due  to  the  significance  or  severity  of  a  specific 

risk  at  the  One  Book  Chapter  level  and  its  relative  importance  to  the  contract,  facility,  or 
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program  overall.  The  amount  of  rationale  or  detail  for  the  assigned  risk  rating  provided 
at  this  level  is  often  limited  and  simplistic.  Some  merely  provide  scope  descriptions  even 
when  Overall  risk  ratings  are  high.  Others  can  be  quite  thorough  and  provide  sound  and 
meaningful  summaries  for  the  supporting  information  that  follows,  even  when  no 
significant  risk  is  present.  The  length  of  the  plans  varies  as  well  and  there  is  no  clear 
pattern  as  to  this  cause. 

1.  Performance 

While  there  was  no  absolute  majority  for  risk  ratings,  a  significant  and  equal 
proportion  of  the  plans  rated  performance  both  as  a  moderate  and  low  risk  area — 40.5%. 
Performance  can  be  viewed  as  the  riskiest  area  overall,  with  more  high  risk  ratings  than 
schedule  and  cost,  although  not  significantly  so,  16.7%  v.  14.3%  and  1 1.9%  respectively. 

2.  Schedule 

Schedule  closely  resembles  performance  risk  ratings:  40.5%  moderate  risk  and 
42.9%  low.  As  is  often  seen  through  the  study,  performance  and  schedule  more  often 
mirror  each  other  due  to  their  close  relationship  and  ultimate  control  by  the  contractor. 
Poor  or  faulty  performance  will  usually  result  in  schedule  delays  due  to  additional  time 
requirements  arising  from  rework  or  malfeasance.  In  the  reverse,  missed  milestones 
(whatever  the  cause)  reflect  poorly  on  contractor  performance  and  can  often  drive  the  risk 
rating  from  this  vantage. 

3.  Cost 

Cost  risk  was  clearly  the  area  of  least  risk  for  the  plans  overall:  66.7%  rated  cost 
as  a  low  risk  area  with  11.9%  and  19.1%  respectively  rating  cost  as  high  or  moderate. 
Cost  can  remain  isolated  from  performance  and  schedule  difficulties  through  Government 
risk  mitigation  via  selection  of  contract  type  and  payment  terms.  The  Government,  being 
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the  buyer,  has  more  direct  ownership  over  this  area  or  process  than  performance  or 
schedule  and  more  ability  to  dictate  the  final  outcome  ...  at  least  from  a  risk  management 
perspective.  The  Government  doesn’t  perform  the  service  or  manufacture  the  product, 
but  it  does  pay  the  bills. 

The  following  sub-chapters  delineate  risk  ratings  for  each  of  the  One  Book 
Chapters  under  their  cognizant  service  sets.  Contract/contractor  program/facility  specific 
high  risk  areas  are  addressed  in  detail  and  chosen  key  processes/systems  and  their 
associated  risk  handling  tools  are  discussed. 

C.  MAJOR  PROGRAM  RISK  RATING 

The  Major  Program  service  set  employed  in  the  RAMP  database  corresponds  to 

Chapter  2  of  the  One  Book,  Major  Program  Services.  Two  of  the  six  subchapters.  Earned 
Value  Management  and  Acquisition  Logistics,  are  available  for  assigning  risk  ratings  in 
RAMP.  Eleven  (11)  of  the  42  sampled  RAMP  plans  rated  risk  areas  for  one  or  more  of 
the  One  Book  Chapters  under  this  service  set.  The  following  risk  management  plans — 
numbered  as  per  Appendix  A — assigned  risk  ratings  in  this  area:  6,  10,  23,  24,  30,  33, 
34,38, 39, 41,  and  42. 

The  individual  ratings  for  performance,  schedule,  and  cost  are  automatically 
generated  for  each  RAMP  plan  based  on  the  input  data  for  all  the  associated  One  Book 
Chapter  risk  ratings  for  each  of  these  areas.  Table  4.2  provides  an  overview  of  the  Major 
Program  service  set  risk  ratings  for  performance,  schedule,  and  cost  of  the  42  sampled 
plans  from  critical  and  strategic  suppliers.  Eleven  (11)  of  the  sampled  plans  addressed 
risk  management  under  the  Major  Program  risk  area.  Thirty-one  (31)  plans  rated  this  risk 
as  not  applicable  and  are  not  depicted  in  the  table. 
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MAJOR  PROGRAM  RISK 

HM 

Mod 

Low 

Performance 

0 

3 

8 

Schedule 

1 

5 

5 

Cost 

2 

1 

8 

Table  4.2.  Major  Program  Service  Set  Risk  Ratings. 
(Source:  Developed  by  Researcher.) 


Major  Program  is  the  least  applied  service  set  among  the  sampled  plans.  Only 
26.2%  of  the  plans  rank  risk  in  this  area  and  most  of  the  risk  was  rated  low:  of  the  plans 
rating  Major  Program  risk,  72.7%  rated  performance  and  cost  risk  as  low,  while  45.5% 
rated  schedule  risk  as  moderate  or  low.  There  were  no  high  risk  ratings  for  performance. 

Of  the  two  assigned  One  Book  Chapters  for  risk  management.  Earned  Value 
Management  was  applied  twice  as  often  as  Acquisition  Logistics  Support:  23.8%  v. 
11.9%  due  to  statement  of  work  (SOW)  requirements  and  Memorandum  of  Agreement 
(MO As)  between  the  buyer  (program  office)  and  the  local  DCMA  office. 

Only  two  plans  (#34  and  #42)  rated  high  risk  at  the  Major  Program  service  set 
level  and  both  were  driven  by  high  risk  ratings  under  Earned  Value  Management  (EVM). 
The  sole  high  risk  rating  for  Acquisition  Logistics  Support  (#41)  was  mitigated  at  the 
Major  Program  level  by  a  low  EVM  value  in  the  same  area.  When  applied,  EVM  seemed 
to  take  a  more  prominent  role  in  the  risk  assignment  for  the  service  set  as  a  whole. 

1.  Earned  Value  Management 

The  supplier  uses  an  Earned  Value  Management  System  (EVMS)  to  provide 

management  information  on  technical  performance,  schedule,  and  cost.  They  must 

ensure  compliance  with  industry  guidelines  and  contract  requirements.  As  part  of  its  risk 

management  efforts,  DCMA  must  provide  EVMS  system  surveillance  and  program 

analysis  to  its  customers.  Table  4.3  provides  an  overview  of  the  key  processes/systems 

45 


chosen  for  risk  management  efforts  under  the  Earned  Value  Management  One  Book 
Chapter  2.2. 


EARNED  VALUE 

MANAGEMENT 

RISK  MANAGEMENT 
PLAN  NO.s 

KEY  PROCESSES/SYSTEMS  6  10  23  24  30  33  34  38  41  42 

r 

r 

r 

r 

n 

Accounting 

IB 

IB 

IB 

IB 

IB 

Analysis 

■ 

IB 

IB 

Baselining  Changes 

X 

r 

IB 

IB 

Budgeting 

IB 

IB 

Change  Incorporation 

IB 

IB 

IB 

Cost  Performance  Report 

a 

■ 

r 

Cost  Variance 

IB 

B 

Cost/Schedule  Variance 

B 

a 

B 

Estimate  at  Completion  (EAC) 

a 

m 

B 

Earned  Value  (EV) 

X 

Forecasting 

X 

Indirect  Management 

a 

Material  Management 

a 

Management  Analysis 

■1 

□i 

a 

a 

a 

X 

Management  Reserve 

■i 

■i 

m 

Bl 

X 

Organizing 

m 

a 

Bl 

Bl 

Bl 

Schedule  Variance 

Bl 

Bl 

Bl 

Bl 

Scheduling 

□1 

X 

Bl 

X 

Subcontract  Management 

■i 

Bl 

X 

Training 

Bl 

Bl 

Undistributed  Budgeting 

Bl 

Bl 

Bl 

Use  of  EV  Data 

X 

Work/Budget  Authorization 

□1 

■ 

Bl 

Table  4.3.  Key  Processes/Systems  for  Earned  Value  Management. 

(Source:  Developed  by  Researcher.) 

Ten  (10)  of  42  RAMP  plans  (or  23.8%  of  the  plans  sampled)  addressed  risk  for 
EVMS.  The  plans  focused  on  23  different  key  processes/systems  and  used  ten  (10) 
different  combinations  of  processes  and  systems  within  EVMS  to  assess  risk  for  the 
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contractor,  facility,  or  contract  in  question.  As  no  two  plans  are  alike  in  the  specific 
processes  or  systems  they  survey,  it  is  easy  to  conclude  that  risk  management  for  EVMS 
is  very  specific  to  the  contract  in  question.  The  most  prevalent  system  chosen  for  review 
was  “Management  Analysis”,  which  was  chosen  50%  of  the  time  RAMP  plans  addressed 
EVMS. 


Three  plans  rated  EVMS  risk  as  high  in  one  or  more  areas.  The  following  details 
the  high  risk  areas  specific  to  the  plans  indicated  and  their  associated  risk  handling  tools 
chosen  to  mitigate  the  risk: 

•  #23:  The  Army  Tactical  Missile  System  (ATACMS)  Block  II  LRIP 
(low  rate  initial  production)  contract  rated  under  this  plan  for  Lockheed 
Martin  Missiles  and  Fire  Control  is  substantially  behind  schedule  and  has 
caused  the  EVMS  schedule  area  to  be  rated  as  high  risk.  Of  the  two  key 
systems  analyzed  for  EVMS  risk,  “Subcontract  Management”  was  rated  as 
the  high  schedule  risk.  Just  as  contract  clauses  flow  down  to 
subcontractors,  so  does  risk  management.  The  major  subcontractor  has  a 
substantial  negative  schedule  variance  causing  the  high  schedule  risk 
rating  and  additionally  driving  a  moderate  risk  rating  in  the  cost  area  due 
to  the  potential  future  impact  on  cost.  “Data  Analysis”  is  the  selected  risk 
handling  tool  for  “Subcontract  Management”:  Cost/Schedule  Status 
Report  (C/S  SR)  data  from  both  the  prime  and  subcontractor  is  reviewed 
and  analyzed  monthly  to  mitigate  risk.  No  further  risk  handling  detail  was 
provided. 

•  #34:  Honeywell’s  cumulative  cost  variance  for  ten  (10)  out  of  (34) 
WBS  Item  Accounts  is  greater  than  10%  with  a  wide  range  from  +128% 
to  -16%.  Program  costs  are  considered  likely  to  increase  due  to  poor 
control  over  cost  variance  and  threaten  to  drain  the  program  budget  and 
lead  to  the  elimination  of  required  qualification  tests.  For  these  reasons, 
cost  is  rated  as  a  high  risk  area  for  EVMS  and  “Cost/Schedule  Variance” 
is  the  key  system  reviewed  for  risk  management.  Program  funding 
depletion  also  directly  affects  schedule  and  performance  and  drives  their 
moderate  risk  ratings  in  the  EVMS  area.  “Data  Analysis”  is  the  chosen 
risk  handling  tool  for  “Cost/Schedule  Variance”.  Specifically,  a 
remaining  qualification  test  will  be  evaluated  weekly  until  completion. 
Past  program  test  deficiency  causes  will  be  reviewed  to  determine  possible 
preventative  measures  for  corrective  action. 

•  #42:  The  Raytheon  Tucson  Evolved  Sea  Sparrow  Missile  (ESSM)  plan 
rates  schedule  and  cost  as  high  risk  areas  for  Earned  Value  Management 
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leading  to  the  same  risk  rating  at  the  Major  Program  service  set  level  for 
this  plan  as  well.  “Earned  Value”  for  two  contracts  is  the  key  process 
identified  for  risk  management  review.  One  contract  rates  schedule  and 
cost  risk  high  due  to  a  nine-month  negative  schedule  variance  (not 
meeting  delivery  requirements)  and  a  four  month  negative  cost  variance 
($50M+  Over  Target  Baseline).  The  second  contract  rates  schedule  risk 
high  due  to  major  slips  in  key  milestones  or  critical  path  and  high  cost  risk 
due  to  unobtainable  planned  cost  targets  and  regularly  unforeseen  cost 
events  ($50M  Over  Target  Baseline).  “Data  Analysis”  is  the  chosen  risk 
handling  tool  by  and  requires  bi-weekly  reviews  of  Raytheon  Tucson’s 
Cost  Performance  Report  (CPR)  along  with  the  Government’s  Technical 
Representative  weekly  report. 


For  each  of  the  plans  rating  high  risk  for  EVMS,  different  key  processes/systems 
were  chosen  for  risk  management  focus.  The  three  high  risk  plans  used  either  a  sole 
parameter  (#34  and  #42)  or  only  two  areas  to  manage  risk  (#23)  while  the  two  plans  with 
the  greatest  number  of  chosen  key  processes/systems  (#30  and  #41)  ranked  risk  low  in  all 
three  areas.  “Data  Analysis”  was  the  common  tool  used  to  mitigate  high  risk  in  all 
instances,  but  different  data  sources  were  identified  for  each  of  the  three  high  risk  plans 
to  mitigate  risk  and  seemed  appropriate  given  the  differences  in  the  contractor  and 
contractual  arrangements  specific  to  each  plan.  The  lack  of  detailed  rationale  for  risk 
handling  under  “Data  Analysis”  for  “Subcontract  Management”  (#23)  is  understandable 
given  the  indirect  relationship  of  the  Government  to  the  subcontractor.  The  high  risk 
areas  in  #34  and  #42  drive  a  high  risk  rating  at  the  Major  Program  service  set  level. 

2.  Acquisition  Logistics  Support 

DCMA’s  policy  is  to  assess  the  contractor’s  ability  to  meet  technical 
performance,  schedule,  and  cost  goals  for  logistics  support  by  reviewing  progress  on  their 
logistics  activities  and  the  supplier’s  plans,  procedures,  and  reports  representative  of  the 
their  logistics  management  systems/processes.  DCMA  will  identify  problem  areas  and 

recommend  Continuous  Improvement  Opportunities  (CIOs)  or  issue  Corrective  Action 
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Requests  (CARs)  to  affect  process  improvements  to  reduce  total  ownership  cost  (life 
cycle  cost).  Table  4.4  provides  an  overview  of  the  key  processes/systems  chosen  for  risk 
management  efforts  under  the  Acquisition  Logistics  One  Book  Chapter  2.3. 


ACQUISITION  LOGISTICS  SUPPORT 

RISK  MANAGEMENT 
PLAN  NO.s 

6  23  30  39  41 

KEY  PROCESSES/SYSTEMS 

■ 

Cost  As  An  Independent  Variable  (CATV) 

Computer  Resources  Support 

■ 

X 

Depot  Level  Maintenance  Requirements 

□ 

Facilities 

■ 

B 

Logistics  Management  Plan 

X 

B 

B 

Logistics  Demonstration 

B 

Maintainability  Demonstration 

B 

Maintenance  Planning 

X 

Manpower  &  Personnel 

B 

Packaging  &  Handling 

B 

Supply  Support 

B 

■ 

B 

Support  Equipment 

B 

B 

B 

Supportability  Planning 

■ 

B 

B 

Technical  Data 

B 

B 

Training  and  Support 

X 

U 

_ 

_ 

_ 

Table  4.4.  Key  Processes/Systems  for  Acquisition  Logistics  Support. 

(Source:  Developed  by  Researcher.) 

Five  of  42  RAMP  plans  (11.9%)  addressed  risk  for  logistics  support.  The  five 
plans  used  15  different  key  processes/systems  in  four  different  combinations  to  assess 
this  risk.  Given  the  low  identification  rate,  logistics  support  does  not  appear  to  be 
recognized  as  a  particularly  risky  area  and  once  identified  there  were  few  similarities  in 
the  systems  or  processes  identified  for  risk  management  efforts.  The  use  of  the 
“Logistics  Management  Plan”  and  “Technical  Data”  were  the  two  most  commonly 
identified  key  processes/systems  for  risk  management  of  contractor  logistics  support. 


used  in  three  of  five  instances. 
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Only  one  RAMP  plan  of  the  42  sampled  plans  rated  high  risk  in  this  area: 

•  #41:  Raytheon  Tucson  Systems  plan  rated  schedule  risk  high  and 

performance  and  cost  risk  as  moderate.  The  high  risk  rating  is  supported 
by  the  “Supply  Support”  and  “Support  Equipment”  key  processes/system 
which  indicate  contractor  schedule  slippages  are  due  to  lack  of  master 
scheduling.  This,  in  addition  to  adverse  performance  trends,  drive 
anticipated  delays  in  meeting  performance,  schedule,  and  cost  objectives. 
100%  monthly  and  quarterly  “Data  Analysis”  is  the  chosen  risk  handling 
tool  and  includes  a  review  of  the  following  data  sources:  schedule 
analysis,  delivery  trend  analysis  program  review,  root  cause  data,  and  cost 
performance  data. 


This  one  high  risk  plan  uses  ten  different  key  processes/systems  to  manage 
contractor  risk  but  most  of  these  areas  are  not  yet  actually  rated  and  remain  “in  process”. 
The  chosen  key  processes  are  consistent  with  the  Acquisition  Logistics  Support  chapter 
and  the  risk  assignments  are  adequately  supported  by  rationale  and  clearly  linked  with 
each  other.  The  risk  handling  tool,  “Data  Analysis’  is  consistent  with  the  trend  in  the 
Major  Program  risk  area  and  is  appropriately  detailed  in  the  RAMP  plan  as  to  the 
specifics  of  the  data  review.  The  high  risk  here  is  mitigated  at  the  Major  Program  level 
by  a  lower  risk  under  EVM. 

D.  PRODUCT  SUPPORT  RISK  RATING 

The  Product  Support  service  set  employed  in  the  RAMP  database  corresponds  to 

Chapter  4  of  the  One  Book,  Product  Performance  Services  -  Right  Item.  Eight  of  the  ten 

subchapters  are  available  for  assigning  risk  ratings  in  RAMP:  Systems  Planning, 

Research,  Development  and  Engineering  (SPRD&E),  Test  and  Evaluation  Management, 

Configuration  Management,  Parts  Management  Program,  Software  Contract 

Administration  Services,  Supplier  Quality  Assurance,  and  Packaging  Management 

Program.  Two  of  these  subchapters  are  further  broken  down:  SPRD&E  -  Design 

Engineering  and  SPRD&E  —  Systems  Engineering;  Supplier  Quality  Assurance  —  Quality 
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System  and  Supplier  Quality  Assurance  -  Product  Quality.  Thirty-six  (36)  of  the  42 
sampled  RAMP  plans  rated  risk  areas  for  one  or  more  of  the  One  Book  Chapters  under 
this  service  set.  The  following  risk  management  plans — numbered  as  per  Appendix  A — 
assigned  risk  ratings:  1  -  9,  1 1  -  26, 29  -  3 1 , 33  -  39, 41 ,  and  42. 

The  individual  ratings  for  performance,  schedule,  and  cost  are  automatically 
generated  for  each  RAMP  plan  based  on  the  input  data  for  all  the  associated  One  Book 
Chapter  risk  ratings  for  each  of  these  areas.  Table  4.5  provides  an  overview  of  the 
service  set  risk  ratings  in  performance,  schedule,  and  cost  of  the  42  sampled  plans  from 
critical  and  strategic  suppliers.  Thirty-five  (35)  of  the  sampled  plans  addressed  risk 
management  under  the  Product  Support  risk  area.  Two  plans  are  “in  process”  of 
assigning  risk  ratings  and  five  plans  rated  this  risk  as  not  applicable;  these  seven  plans  are 
not  depicted  in  the  table. 


PRODUCT  SUPPORT  RISK 

High 

Mod 

Low 

Performance 

3 

14 

18 

Schedule 

5 

9 

21 

Cost 

2 

11 

22 

Table  4.5.  Overview  of  the  Service  Set  Risk  Ratings  for  Product  Support. 

(Source:  Developed  by  Researcher.) 

Product  Support  is  the  most  applied  service  set  among  the  sampled  plans.  A 
strong  absolute  majority  of  83.3%  of  the  plans  rank  risk  in  this  area.  It  is  the  largest  area 
with  the  largest  scope  from  the  standpoint  of  using  seven  different  One  Book  Chapters 
and  nine  different  risk  management  areas  (two  of  the  chapters  being  split  into  two  areas). 
Despite  the  size  and  potential  for  risk,  given  the  subject  area  of  the  service  set,  risk 
remained  low:  62.9%  and  60%  of  the  plans  respectively  ranked  cost  and  schedule  risk  as 
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low,  while  still  a  clear  majority  of  51.4%  ranked  performance  risk  low  and  40%  ranked 
performance  risk  as  moderate. 

Of  the  nine  One  Book  Chapter  applications,  Supplier  Quality  Assurance  -  Product 
Quality  was  used  two  to  five  times  as  often  as  any  other  One  Book  Chapter  level  area.  It 
was  the  most  commonly  used  ranking  area  of  any  in  the  RAMP  program:  73.8%  of  the 
RAMP  plans  addressed  risk  in  this  area.  Even  when  only  one  or  two  areas  are  ranked 
under  Product  Support,  Supplier  Quality  Assurance  -  Product  Quality  remains  the  key 
chosen  factor.  Given  the  area’s  broad  scope  and  clear  application  in  the  post-award 
contract  phase  of  acquisition,  this  is  not  surprising.  Due  to  its  frequency  of  use  both 
when  few  and  many  One  Book  Chapters  are  selected  for  risk  management,  it  is  the  key 
driving  factor  in  the  overall  risk  ratings  at  the  Product  Support  service  set  level;  although 
when  include  with  others,  it’s  ratings  do  not  seem  to  out  weigh  the  other  applications. 

Only  three  plans  (#17,  #20,  and  #37)  rated  high  risk  at  the  Product  Support 
service  set  level  and  all  three  were  strongly  driven  by  high  risk  rankings  for  Product 
Quality. 

1.  SPRD&E  -  Design  Engineering 

SPRD&E  surveillance  is  a  risk  assessment  of  the  suppliers  to  conduct  systems 
planning,  research,  development  and  engineering  including  engineering  systems, 
processes,  policies,  procedures,  practices,  activities,  and  products.  The  DCMA  focus 
here  is  on  design  engineering  to  ensure  compliance  with  contract  requirements  as 
affecting  technical  performance,  schedule,  and  life  cycle  cost.  Table  4.6  provides  an 
overview  of  the  key  processes/systems  chosen  for  risk  management  efforts  under  the 
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System  Planning,  Research,  Development  and  Engineering  (SPRD&E)  One  Book 
Chapter  4.1. 


SPRD&E  -  DESIGN  ENGINEERING 

RISK  MANAGEMENT 

PLAN  NO.s 

KEY  PROCESSES/SYSTEMS 

1 

6 

22  26  30  38  42 

Cost  Proposal  Analysis 

X 

Design  Analysis 

X 

X 

Design  Review 

X 

Deviations/Waivers/Engineering 
Change  Proposal  (ECP)  Evaluations 

X 

Engineering  Planning 

X 

Engineering  Management 

X 

X 

X 

X 

X 

Producibility 

X 

Software 

X 

Systems  Design 

X 

Table  4.6.  Key  Processes/Systems  for  SPRD&E-Design  Engineering. 

(Source:  Developed  by  Researcher.) 

Seven  of  42  RAMP  plans  surveyed  (16.7%)  contractor  design  engineering  efforts 
as  part  of  their  product  support  efforts.  The  seven  plans  used  nine  different  key 
processes/systems  in  seven  different  combinations  to  assess  this  risk.  While  there  was 
clearly  a  lot  of  variation  in  the  key  processes/systems  used  by  the  plans,  Engineering 
Management  was  clearly  the  most  prevalent  process  identified  for  risk  management,  used 
71.4%  of  the  time. 

There  were  two  instances  of  high  risk  ratings  for  design  engineering  efforts.  The 
following  details  the  specifics  for  the  applicable  plans  and  discusses  their  chosen  risk 
handling  methods: 

•  #30:  Aerojet  was  assigned  a  high  risk  rating  for  design  engineering  in 

all  three  areas  of  performance,  schedule,  and  cost.  “Design 
Analysis/Synthesis”  was  the  chosen  key  process/system  for  Government 
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surveillance.  Improper  design  requirements  could  lead  to  incorrect  design 
solutions  and/or  environmentally  hazardous  conditions  impacting  cost, 
schedule,  and  performance.  Although  the  probability  of  occurrence  was 
only  rated  moderate,  the  consequence  of  occurrence  was  rated  high  in  that 
failure  could  likely  result  in  mission  failure.  The  chosen  risk  handling 
methods  were  “Surveillance”  and  “Data  Analysis”.  Specifically, 
requirements  analysis,  functional  analysis/allocation,  and  synthesis 
processes  were  monitored;  various  activities  and  metrics  were  surveyed; 
and  policies  and  procedures  were  reviewed.  The  surveillance  revealed  no 
systemic  problems  and  overall  contractor  performance  was  considered 
good,  but  the  current  risk  ratings  and  handling  methods  will  remain  in 
place  due  to  the  high  risk  of  consequence  should  failure  occur. 

•  #42:  Raytheon  Tucson  ESSM  program  rated  schedule  and  cost  as  high 

risk  areas  under  design  engineering.  One  contract  for  an  Engineering 
Manufacturing  Development  (EMD)  program  identified  “Engineering 
Management”  and  “Engineering  Planning”  as  key  processes/systems  to 
use  for  risk  management.  Failure  to  properly  control  either  area  has  the 
potential  to  impact  cost  and  schedule  and  future  transitioning  into 
production.  A  major  slip  in  key  milestones  and  critical  path  has  led  to  a 
schedule  extension.  Cost  is  rated  high  due  to  unobtainable  cost  targets  and 
regularly  unforeseen  cost  events;  the  contract  is  $50M+  over  contract 
value.  A  second  contract  for  Low  Rate  Initial  Production  (LRIP)  program 
identified  “Producibility”  and  “Software”  as  key  processes/system  to  use 
for  risk  management.  A  nine  month  negative  schedule  variance  and 
difficulty  in  meeting  delivery  requirements  drives  a  high  risk  rating  for 
schedule.  Cost  is  rated  high  to  the  contractor’s  failure  to  contain  costs;  the 
contract  is  $50M+  Over  Target  Baseline  (OTB)  and  has  a  four  month 
negative  cost  variance.  Ratings  for  “Software”  are  in  process.  “Data 
Analysis”  was  the  chosen  risk  handling  tools  for  the  three  rated  processes: 
The  contractors  Cost  Performance  Report  (CPR),  Cost  Schedule/Status 
Report  (CSSR),  and  the  Government’s  Technical  Representative  weekly 
report  will  be  reviewed  monthly. 

For  both  of  the  plans  rating  high  risk  for  Design  Engineering,  completely  different 
key  processes/systems  were  chosen  for  risk  management  focus.  Plan  #  30  focused  on 
only  one  key  parameter — “Design  Analysis”,  while  plan  #42  used  a  mixture  of  four 
different  plans  to  manage  risk.  These  differences  are  consistent  with  the  contractual 
efforts  and  well  explained  and  documented  in  the  rationale.  Plan  #30  is  focused  on  the 
potential  impact  of  the  risk  vice  its  low  probability  of  occurrence  (the  contractor  has 
demonstrated  good  performance);  plan  #42  rated  risk  under  two  different  contracts  in 


54 


different  acquisition  phases:  the  engineering  based  processes  were  chosen  for  risk 
management  during  EMD  and  production  based  processes  were  used  during  LRIP  (the 
contractor  is  already  experiencing  some  difficulty  in  fulfilling  contractual  requirements 
and  requires  a  different  focus).  “Data  Analysis”  is  once  again  chosen  as  a  risk  mitigation 
tool  for  both  plans,  specific  to  the  data  for  the  processes  chosen.  “Surveillance”  is 
additionally  used  in  plan  #30  and  is  used  to  assess  the  metrics  and  processes  for  systemic 
difficulties.  Finding  none  and  given  the  contractor’s  performance,  it  appears  to  be  a 
worthy  task  to  eliminate  probability  concerns  and  focus  instead  on  mitigation  of  impact 
of  failure.  The  high  risk  for  both  plans  in  the  Design  Engineering  area  is  not  driving  high 
risk  ratings  for  the  Product  Support  service  set  level;  each  plan  uses  five  other  various 
One  Book  Chapter  areas  for  risk  management  under  Product  Support  and  successfully 
mitigates  the  service  set  level  risk  rating. 

2.  SPRD&E  -  Systems  Engineering 

SPRD&E  surveillance  is  a  risk  assessment  of  the  suppliers  to  conduct  systems 
planning,  research,  development  and  engineering  including  engineering  systems, 
processes,  policies,  procedures,  practices,  activities,  and  products.  The  DCMA  focus 
here  is  on  engineering  management  systems  to  ensure  compliance  with  contract 
requirements  as  affecting  technical  performance,  schedule,  and  life  cycle  cost.  Table  4.7 
provides  an  overview  of  the  key  processes/systems  chosen  for  risk  management  efforts 
under  the  Systems  Planning,  Research,  Development  and  Engineering  (SPRD&E)  One 
Book  Chapter  4.1. 
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SPRD&E  -  SYSTEMS 

ENGINEERING 

RISK  MANAGEMENT 
PLAN  NO.s 

KEY  PROCESSES/SYSTEMS  22  23  24  26  30  33  35  38  39  41  42 
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■ 

■ 

m 

fl 

■ 

■ 

■ 

B 

B 

Design  Engineering 

■ 

ID 

■ 

■ 

■ 

■ 

■ 

■ 

B 

B 

Detail  Design 
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IB 

fl 
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■ 

IB 
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d 

D 
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D 

D 

■ 

D 

■ 
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B 

Requirements  Analysis 
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■ 
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■ 

D 

B 

B 

fl 

Resource  Management 

fl 
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□ 

fl 

Subcontractor  Engineering  Design 

D 

n 
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■ 

D 

■ 

■ 
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B 

fl 
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Systems  Design 

■ 

D 
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B 

fl 

fl 

B 

Systems  Engineering 
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D 

■ 

S3 
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B 

B 
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D 

Bi 

B 

B 

fl 
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□ 

■ 

fl 

fl 

B 
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D 

D 

□ 
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n 

Z1 
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BI 

BI 

B 
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bi 
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■ 

BI 

BI 
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■ 

BI 

BI 

B 

■1 

Technical  Performance 

□ 

3 

□ 

Table  4.7.  Key  Processes/Systems  for  (SPRD&E)-Systems  Engineering. 

(Source:  Developed  by  Researcher.) 

Eleven  (11)  of  42  RAMP  plans  (26.2%)  surveyed  contractor  systems  engineering 
efforts  as  part  of  their  product  support  efforts.  The  11  plans  used  25  different  key 
processes/systems  in  1 1  different  combinations  to  assess  this  risk.  There  is  clearly  a  lot 
of  variation  in  the  choice  of  key  systems/processes  to  use  for  risk  management.  The  most 
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often  used  were  Systems  Engineering,  Systems  Requirements,  and  Systems  Safety,  each 
used  in  36.4%  of  the  plans. 

There  was  one  instance  of  high  risk  rating  for  systems  engineering.  The 
following  details  the  specifics  of  the  applicable  plan  and  discusses  its  chosen  risk 
handling  method: 

•  #42:  Raytheon  Tucson  ESSM  program  rated  schedule  and  cost  as  high 

risk  areas  for  systems  engineering.  “Systems  Requirements”  was  chosen 
as  the  key  process  for  risk  management  under  the  EMD  program  contract. 
A  major  slip  in  key  milestones  and  critical  path  has  led  to  a  schedule 
extension  and  the  high  schedule  risk  rating.  Cost  is  rated  high  due  to 
unobtainable  cost  targets  and  regularly  unforeseen  cost  events;  the 
contract  is  $50M+  over  contract  value.  “Data  Analysis”  is  the  chosen  risk 
handling  tool:  a  bi-weekly  review  of  CPR  along  with  the  Government 
Technical  Representative  weekly  report. 

This  one  high  risk  plan  in  this  area  uses  only  one  key  system,  “Systems 
Requirements”  to  manage  contractor  risk  while  the  other  ten  less  risky  plans  used  an 
average  of  four  key  parameters  each  to  mitigate  risk  in  the  Systems  Engineering  area. 
The  rationale  seems  supportive  of  the  assigned  rating  and  consistent  with  the  EMD 
design  and  integration  activities  that  have  experienced  technical  difficulties.  The  chosen 
risk  handling  tool  is  “Data  Analysis”  of  performance  reports  applicable  to  the 
contractor’s  requirements. 

3.  Test  and  Evaluation 

The  focus  here  is  on  the  manufacturer’s  test  engineering/design  process  and  test 
management  systems  that  verify  compliance  with  contract  performance  requirements. 
DCMA  seeks  to  identify  potential  test  problems  and  notify  customers  of  suppler  test 
decisions.  Test  data  can  be  an  indicator  of  supplier  problems  in  design,  development, 
production,  or  system  deployment.  Table  4.8  provides  an  overview  of  the  key 
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processes/systems  chosen  for  risk  management  efforts  under  the  Test  and  Evaluation 
Management  One  Book  Chapter  4.1.1. 


TEST  &  EVALUATION 

RISK  MANAGEMENT 

PLAN  NO.s 

KEY  PROCESSES/SYSTEMS  1  6  22  23  24  26  30  33  34  35  38  39  41  42 
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fl 
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fl 

B 

B 
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□ 

Technical  Performance 
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fl 
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X 

X 
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X 
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Bi 

B 
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Bl 
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Bl 

Bl 

Bl 
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Bl 

fl 
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□ 
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□ 
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□ 
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Table  4.8.  Key  Processes/Systems  for  Test  and  Evaluation. 
(Source:  Developed  by  Researcher.) 


Fourteen  (14)  of  42  RAMP  plans  (33.3%)  assessed  supplier  test  and  evaluation 
performance.  Eighteen  (18)  key  processes/sy stems  in  13  different  combinations  are  used 
to  assess  contractor  risk  in  the  14  plans.  There  is  a  great  deal  of  variability  in  the  chosen 
key  systems/processes  for  risk  management  efforts.  However,  “Acceptance  Tests”  were 
used  in  35.7%  of  the  plans  rating  the  test  and  evaluation  efforts.  No  risk  areas  were  rated 
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high.  Seven  plans  ranked  risk  as  low,  six  ranked  risk  as  medium,  and  one  plan  remained 
“in  process”. 

4.  Configuration  and  Technical  Data  Management 

The  contractor  conducts  Configuration  and  Technical  Data  Management  to 

maintain  product  design  and  integrity;  control  form,  fit,  and  function;  determine 

engineering  and  cost  tradeoff  decisions  of  technical  performance,  producibility, 

operability,  and  supportability;  and  maintain  historic  data  files.  DCMA’s  role  is  to  verify 

the  contractor’s  process  has  controls  for  establishing  the  proper  baseline  and  perform 
* 

necessary  reviews  and  product  audits  to  ensure  the  contractor’s  compliance.  Table  4.9 
provides  an  overview  of  the  key  processes/systems  chosen  for  risk  management  efforts 
under  the  Configuration  Management  One  Book  Chapter  4.2. 

Sixteen  (16)  of  the  42  RAMP  plans  (38.1%)  evaluated  this  area  for  risk.  Thirteen 
(13)  key  processes/systems  in  nine  different  combinations  are  used  to  assess  contractor 
risk  in  the  16  plans.  While  there  is  a  lot  of  variability  between  the  plans  as  a  whole, 
“Configuration  Control”  was  used  in  56.3%  of  the  plans  rating  configuration  and 
technical  data  management.  There  were  no  high  risk  areas  and  risk  throughout  was 
predominantly  low:  68.8%  ranked  performance  and  schedule  risk  as  low;  75%  ranked 
cost  risk  low. 
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CONFIGURATION  & 

TECHNICAL  DATA 

MANAGEMENT 

RISK  MANAGEMENT 

PLAN  NO.s 

KEY  PROCESSES/SYSTEMS  1  2  3  5  6  22  23  24  26  33  34  35  38  39  41  42 
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a 

Configuration  Identification 

□ 

B 

B 

a 

a 

Data  Management 

□ 

B 
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□ 

■ 
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n 
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1 

1 

1 

1 

■ 

■ 

■ 

i 

■ 

i 

X 

i 

■ 

n 

Nonconforming  Material/Material 
Review  Board  (MRB) 

X 

Value  Engineering  Incentives 

X 

Table  4.9.  Key  Processes/Systems  for  Configuration  and  Technical  Data 

Management. 

(Source:  Developed  by  Researcher.) 

5.  Parts  Management  Program 

The  Parts  Management  Program  is  intended  to  standardize  parts  to  reduce 
inventory  and  costs  for  drawings  and  testing  and  improve  systems  commonality, 
interoperability,  reliability,  standardization,  maintainability,  and  interchangeability. 
DCMA  must  assess  the  contractor’s  program  in  this  area  to  account  for  risk  associated 
with  noncompliance  and  possible  impact  to  performance,  schedule,  and  cost.  Table  4.10 
provides  an  overview  of  the  key  processes/sy stems  chosen  for  risk  management  efforts 
under  the  Parts  Management  Program  One  Book  Chapter  4.2.1. 

Six  of  42  RAMP  plans  (14.3%)  address  this  risk  area.  Fourteen  (14)  key 
processes/systems  are  used  in  six  different  combinations  to  assess  contractor  risk  in  the 
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six  plans.  “Parts  Evaluation/Authorization  Process”  is  the  only  key  process  identified  as 
a  risk  management  area  in  two  separate  RAMP  plans.  There  is  absolutely  no 
commonality  here  between  frequency  of  chosen  processes/systems  for  risk  management 
or  overall  configuration  of  the  risk  management  plan. 


PARTS  MANAGEMENT 

RISK  MANAGEMENT 
PLANS  NO.s 

22  23  24  33  34  41 

PROGRAM 

KEY  PROCESSES/SYSTEMS 

Assess  Parts  Suppliers 

X 

Design  and  Requirements  Process 

X 

GIDEP  Alerts 

X 

GIDEP/DMSMD/MPCASS 

X 

Handling 

Marking 

Nonstandard  Parts 

X 

Packaging 

Parts  Evaluation/Authorization 
Process 

X 

X 

Parts  List  Tracking 

X 

Parts  Management  Plan 

X 

Safety  of  Flight  Parts 

X 

Subcontractors 

X 

Supplier  Policies,  Procedures, 
Practices 

X 

Table  4. 1 0.  Key  Processes/Systems  for  Parts  Management  Program. 

(Source:  Developed  by  Researcher.) 


There  was  one  instance  of  high  risk  in  Parts  Management  Program.  The 
following  details  the  specifics  for  the  applicable  plan  and  discusses  its  chosen  risk 
handling  methods: 

•  #22:  McDonnell  Douglas  Helicopter  Systems’  Parts  Management 

Program  was  rated  as  a  high  performance,  schedule,  and  cost  risk.  High 
performance  risk  is  due  to  high  consequence  of  failure,  outstanding 
corrective  action  issues  from  a  previous  audit,  and  past  performance 
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instances  of  similar  corrective  action  difficulties.  High  schedule  risk  is 
based  on  the  consequences  of  process  failure  and  previous  contractor 
mitigation  factors.  High  cost  risk  is  present  due  to  consequences  despite 
the  fact  the  contractor  has  adequate  processes  in  place.  “Safety  of  Flight 
Parts”  is  the  key  process/system  identified  for  risk  management  based  on 
the  safety  and  mission  elements  it  controls.  Monthly  “Data  Analysis”  of 
metrics  and  100%  inspections  of  all  flight  safety  part  installations  are  the 
chosen  risk  handling  methods  and  are  conducted  dually  by  DCMA  and 
Boeing. 

Parts  Management  is  the  least  used  of  the  nine  areas  under  Product  Support  but  its 
risk  factors  for  the  plans  overall  do  not  indicate  any  nuances  different  from  the  other 
areas.  The  one  high  risk  plan  manages  contractor  risk  through  the  use  of  one  key  process, 
“Safety  of  Flight  Parts”  which  is  a  proper  focal  point  for  mitigating  impact  of  failure, 
which  is  loss  of  life  in  this  case.  “Data  Analysis”  of  metrics  and  100%  inspections  seems 
appropriate  given  the  nature  of  the  risk  and  chosen  key  parameter  for  risk  management. 
There  is  no  significant  correlation  between  risk  ratings  under  this  area  and  those  derived 
at  the  service  set  level  other  than  contributing  factors  to  the  average  rating. 

6.  Software  CAS 

Software  Configuration  Management  Services  include  software;  the  supplies, 
processes,  procedures,  and  activities  attributable  to  software  development;  software 
documentation;  software  embedded  in  test  equipment;  and  non-deliverable  software 
products.  DCMA  assess  the  contractor’s  software  development  efforts  and  possible 
performance,  schedule,  and  cost  impacts.  Table  4.11  provides  an  overview  of  the  key 
processes/systems  chosen  for  risk  management  efforts  under  the  Software  Contract 
Administration  Services  (CAS)  One  Book  Chapter  4.3. 

Eleven  of  42  RAMP  plans  (26.2%)  provide  for  contractor  surveillance  in  this 
area.  Seventeen  (17)  key  processes/systems  are  identified  and  ten  combinations  used  for 
the  eleven  plans.  “Software  Configuration  Management”  is  the  most  frequently  used  key 
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process/system,  applied  81.8%  of  the  time.  “Software  Quality  Assurance”  likewise  is 
used  to  a  significant  degree,  63.6%  of  the  time.  There  is  clearly  a  lot  of  commonality  and 
congruence  between  the  various  plans  under  Software  CAS. 


SOFTWARE  (SW)  CAS 

RISK  MANAGEMENT 
PLANS  NO.s 

KEY  PROCESSES/SYSTEMS  1  4  6  20  23  24  30  3539  41  42 
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B 

SW  Product  Engineering 

B 

X 

X 

X 

SW  Project  Planning 

B 

B 

B 

B 

X 

X 
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Table  4.11.  Key  Processes/Systems  for  Software  CAS. 

(Source:  Developed  by  Researcher.) 

There  is  only  one  incidence  of  high  risk  in  Software  CAS.  The  following  details 
the  specifics  for  the  applicable  plan  and  discusses  its  chosen  risk  handling  methods: 

•  #20:  Motorola  SSG  received  a  high  risk  rating  for  schedule  in  Software 

CAS.  This  is  because  Motorola  is  going  to  deliver  six  to  eight  months 
late.  Despite  this  the  cost  risk  remains  low,  as  this  is  a  fixed  price 
contract.  “Software  Project  Planning”  is  the  key  process  identified  for  risk 
management;  this  monitors  the  contractor’s  compliance  to  the  applicable 
software  development  plan  and  Statement  of  Work  (SOW)  which  remains 
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at  risk  due  to  inadequately  defined  interfaces  slowing  the  software 
development  and  affecting  schedule.  “CMM  Based  Insight”  (contractor 
monitoring)  is  the  identified  risk  handling  method;  this  includes  a  review 
of  the  contractor’s  software  development  plan  (SDP)  and  the  statement  of 
work  (SOW). 

Most  of  the  risk  under  Software  CAS  is  low:  70.0%  each  for  performance  and 
schedule  and  90%  for  cost.  The  one  plan  rating  high  risk  for  software  (#20)  uses  only 
one  key  process  for  risk  management:  “Software  Project  Planning”  which  is  not  used 
solely  elsewhere,  but  only  in  conjunction  with  other  processes/systems  to  manage 
software  risk  in  totality.  It  was  chosen  in  this  case  because  it  is  a  Letter  of  Delegation 
(LOD)  task  and  it  appears  to  be  consistent  with  a  subcontracted  effort.  The  high  schedule 
risk  under  Software  CAS  for  plan  #20  is  a  contributing  factor  to  the  high  risk  rating 
assigned  to  schedule  at  the  Product  Support  service  set  level  for  plan  #20,  but  as  with  the 
other  plans,  it  does  not  seem  to  be  an  overriding  factor.  Monitoring  contractor  meetings 
(“CMM  Based  Insight”)  as  the  chosen  risk  handling  tool  seems  appropriate  as  well  to  a 
subcontract  effort. 

Current  information  indicates  Program  Managers  consistently  have  problems  with 
software  acquisition  in  the  form  of  cost  overruns,  slippage  in  schedule,  and 
nonperformance  in  terms  of  meeting  specification  standards,  mission  requirements,  and 
functionality.  (Nissen)  In  fact,  it  is  often  regarded  as  the  highest  risk  element  in  weapon 
system  development:  management  is  inconsistent  or  reactive,  predictable  risks  are 
ignored,  and  quality  standards  are  often  traded  for  schedule,  performance,  or  cost. 
(GSAM  6.4. 1.1,  2000)  Given  this,  it  is  surprising  DCMA  rates  risk  in  this  area  so  low; 
they  may  be  underestimating  the  probability  or  magnitude  of  the  problem  should  software 
development  go  awry. 
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7.  Supplier  Quality  Assurance  -  Quality  System 

DCMA  performs  oversight  functions  to  assess  contractor  compliance  with 
technical,  manufacturing,  and  quality  assurance  requirements.  Due  to  the  breadth  of  this 
program,  multi-functional  teams  often  perform  the  surveillance  to  maximize  the  scope  of 
the  evaluation  and  share  information  within  the  DCMA  Contract  Management  Office 
(CMO).  Quality  Assurance  activities  are  •  performed  whenever  inspection  and/or 
acceptance  at  origin  is  assigned  to  DCMA  unless  specifically  not  required  by  the 
customer  or  governed  by  other  policies.  For  RAMP  purposes,  Supplier  Quality 
Assurance  is  divided  into  two  areas  for  risk  handling:  Quality  System  and  Product 
Quality. 

Quality  System  audits  are  performed  when  directed  by  the  customer,  existing  data 
is  inadequate  or  unavailable  to  properly  assess  the  contractor  quality  assurance  system,  or 
the  contractor’s  process  has  been  substantially  changed,  requiring  a  new  baseline  review. 
DCMA  measures  performance  against  the  DCMA  Audit  Checklist  and  International 
Organization  for  Standardization  (ISO)  9000  series  quality  systems  models.  The 
contractor  is  invited  to  participate  in  these  audits.  Table  4. 1 2  provides  an  overview  of  the 
key  processes/systems  chosen  for  quality  system  risk  management  efforts  under  the 
Supplier  Quality  Assurance  (QA)  One  Book  Chapter  4.4. 

Fourteen  (14)  of  42  RAMP  plans  (33.3%)  addressed  quality  systems  as  a  risk 
management  area.  The  plans  focused  on  26  different  key  processes/system  and  used  13 
different  combinations  of  processes  and  systems  within  quality  systems  to  assess  risk  for 
the  contractor,  facility,  or  contract  in  question.  The  most  prevalent  systems  chosen  for 
review  were  “ISO  9002”,  “Design  Control”,  and  “Internal  Quality  Audits”;  each  used 
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21.4%  of  time.  As  is  visibly  apparent  in  the  table  there  is  no  real  commonality  or 
congruence  between  the  plans  with  most  key  processes/systems  only  being  used  once. 
Additionally,  one  plan,  Raytheon  Tucson  Systems  (#41)  clearly  addresses  risk 
management  to  a  degree  not  even  approached  in  the  other  13  plans. 

One  RAMP  plan  rated  quality  system  risk  as  high.  The  following  details  the  high 
risk  rating  in  all  three  areas  of  performance,  schedule,  and  cost  and  the  associated  risk 
handling  tool  chosen  to  mitigate  the  risk: 

•  #20:  DCMA  rates  performance,  schedule,  and  cost  as  high  risk  areas  for 

the  Motorola  SSG  plan  citing  ,a  near  certainty  of  complete  failure  for  sub¬ 
system  of  the  F-22  program.  Significant  instances  where  there  are  product 
quality  issues  for  form,  fit,  and  function  and  resource  deficiencies  in  the 
form  of  new  employees/engineers  drive  the  poor  performance  rating.  The 
schedule  for  estimated  time  of  delivery  has  already  been  extended  five 
months  beyond  purchase  order  delivery  date.  The  cost  is  likewise  rated 
high,  even  though  this  is  a  fixed-price  subcontract  due  to  the  high 
probability  of  unknowns  becoming  out-of-scope  work  issues.  “Design 
Control”  is  the  key  process  chosen  for  risk  management  and  “System 
Evaluation”  is  the  risk  handling  tool:  DCMA  QA  Representative  is  to 
attend  the  bi-weekly  meeting  with  the  Quality  Assurance  Team  for 
problem  status  and  schedule  impact. 

The  one  high  risk  plan  uses  only  one  key  process/system  out  of  26  different  options  used 

throughout  the  sample  for  risk  management  in  the  Quality  System  area:  “Design 

Control”.  All  three  risk  areas  of  performance,  schedule,  and  cost  here  drive  higher  risk 

ratings  at  the  Product  Support  service  set  level  for  plan  #20  and  high  risk  for  the  plan 

overall.  This  high  risk  plan  is  not  inconsistent  with  the  other  plans  applying  Quality 

System  risk  management  efforts;  64.3%  of  the  plans  use  only  one  key  process/system  for 

risk  mitigation  efforts.  However,  risk  in  this  area  generally  remains  low:  68.8%  rate 

schedule  and  cost  risk  as  low;  56.3%  rate  performance  risk  low.  The  risk  under  plan  #20 

however,  runs  consistent  throughout  the  entire  RAMP  plan  and  draws  a  common  thread 

through  the  other  two  One  Book  Chapter  areas  under  which  it  addresses  risk 
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management.  The  chosen  risk  handling  tool  is  “System  Evaluation”  and  this  seems 
consistent  with  the  need  for  frequent  contractor/Govemment  interface. 


SUPPLIER  OA  -  OUALITY 

SYSTEM 

RISK  MANAGEMENT 

PLANS  NO.s 

KEY  PROCESSES/SYSTEMS 

7 

11  12  15  20  22  30  31  33  34  35  36  38  41 

Contract  Review 

X 

Control  Customer  Supply 
Product 

X 

Control  of  Quality  Receipt 

X 

Correct/Prevent  Action 

X 

X 

Control  of  Inspect/Measure/Test 
Equipment 

X 

X 

Control  of  Nonconforming 
Material 

X 

X 

Design  Control 

X 

X 

X 

Document  and  Data  Control 

X 

Handling/Storage/Packaging/ 

Preservation/Delivery 

X 

Inspection  and  Test  Status 

X 

Inspection  and  Testing 

X 

Internal  Quality  Audits 

X 

X 

X 

Into-Plane  Operations 

X 

ISO  9001 

X 

X 

ISO  9002 

X 

X 

X 

Material  Review  Board  (MRB) 

X 

Management  Responsibility 

X 

Prime  Control  of  Sub-Vendors 

X 

Process  Control 

X 

Product 

Identification/r  raceability 

X 

Purchasing 

X 

Quality  System 

X 

X 

Refinery  Operations 

X 

X 

Servicing 

X 

Statistical  Techniques 

X 

Training 

X 

Table  4.12.  Key  Processes/Systems  for  Supplier  QA-Quality  System  Risk 

Management  Efforts. 

(Source:  Developed  by  Researcher.) 
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8.  Supplier  Quality  Assurance  -  Product  Quality 

Each  lot  of  output  from  a  high  risk  processes  must  be  sampled  using  a  statistically 
valid  sampling  plan.  CMO  personnel  have  discretion  in  forming  lots  for  these  samples. 
Table  4.13  provides  an  overview  of  the  key  processes/systems  chosen  for  product  quality 
risk  management  efforts  under  the  Supplier  Quality  Assurance  One  Book  Chapter  4.4. 

Thirty-one  (31)  of  42  RAMP  plans  (73.8%)  addressed  product  quality  as  a  risk 
management  area.  The  plans  used  140  different  key  processes/system  and  29  different 
combinations  of  processes  and  systems  within  product  quality  to  assess  risk  for  the 
contractor,  facility,  or  contract  in  question.  The  most  prevalent  systems  chosen  for 
review  was  “Final  Inspection”,  used  in  23.8%  of  the  plans.  While  this  high  percentage 
might  seem  to  indicate  DCMA  is  waiting  until  the  product  is  finished  before  making  sure 
it  is  acceptable,  the  large  number  of  “in  process”  reviews  (i.e.  the  other  139  key 
processes/systems  used  to  evaluate  product  quality  risk)  and  the  numerous  quality 
assurance  evaluations  of  the  contractors’  Quality  System  cited  earlier  in  this  chapter 
indicates  DCMA’s  proactive  approach  to  monitoring  quality. 

Five  RAMP  plans  rated  product  quality  risk  as  high.  The  following  details  the 
high  risk  rating  for  the  identified  areas  of  performance,  schedule,  and  cost  and  the 
associated  risk  handling  tools  chosen  to  mitigate  the  risk. 
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Table  4. 1 3.  Key  Processes/Systems  for  Supplier  QA-Product  Quality. 
(Source:  Developed  by  Researcher.) 
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#2:  Raytheon  Electronic  Systems  was  rated  with  a  high  performance, 

schedule,  and  cost  risk.  Raytheon  is  currently  unable  to  meet  contracted 
delivery  schedules  due  to  inadequate  manufacturing  capability.  There  was 
no  integrated  master  schedule  between  three  facilities  involved  in  the  F/A 
18  program,  low  yields  on  some  subassembly  circuits,  a  shortage  of  test 
equipment,  and  a  need  for  additional  employees  to  increase  production 
capacity.  Risk  mitigation  plans  were  cited  as  being  in  place.  However, 
the  key  processes/system  identified  for  risk  management  do  not  support 
the  most  recent  rating  assignments.  “Acceptance  Testing”,  “Final 
Inspection”,  and  “Material  Review  Board”  (MRB)  all  contained  the  initial 
low  risk  in  all  three  areas  of  performance,  schedule,  and  cost  and  clearly 
had  not  been  updated  to  support  the  more  current  chapter  rating. 
“Corrective  Action”  (as  required),  “Data  Analysis”  (collected  quarterly) 
and  selected  “Product  Audits”  were  risk  handling  methods  chosen  for  the 
“Acceptance  Testing”  and  “Final  Inspection”  processes.  “Product  Audits” 
are  conducted  for  “use  as  is”  and  “repair”  items  under  the  MRB  process. 

#14:  Westinghouse  Electric  facility  plan  was  rated  as  a  high 
performance,  schedule,  and  cost  risk.  Two  key  processes, 

“Documentation”  and  “Receiving  Inspection”  support  this  risk  ratings 
because  failures  in  these  processes  has  resulted  in  nonconforming  material 
delivered  to  the  customer.  Additionally,  there  are  numerous  contractor 
reorganizations  involving  up  to  50%  personnel  lay-offs  creating  serious 
losses  in  the  corporate  knowledge  base.  “Corrective  Action”  using 
Corrective  Action  Reports  (CARs)  issued  for  contract  deviations,  “Record 
Review”  involving  a  1 00%  review  of  shipment  records  at  final  inspection, 
and  100%  “Product  Audits”  of  all  items  presented  to  the  Government  for 
acceptance  are  the  selected  risk  handling  tools. 

#17:  Stewart  and  Stevenson  were  assigned  high  risk  ratings  for 
performance,  schedule,  and  cost  under  Supply  Quality  Assurance 
Product  Quality.  “Shipping”  is  the  chosen  system  for  risk  management 
review.  The  contractor  has  failed  to  achieve  ISO  9000  certification  and 
was  previously  issued  a  Level  III  Corrective  Action  Request  for 
deficiencies  in  their  quality  management  system.  On-time  delivery  was 
75%  as  result  of  product  quality  deficiencies.  DCM  surveillance  and 
audits  were  suspended  until  the  contractor  can  obtain  a  repeatable  and 
positive  Government  release  quality.  “Inspection”  is  the  chosen  risk 
handling  tool.  There  were  inconsistencies  in  the  rating  narrative:  “Cost” 
was  described  as  low  risk  due  to  fixed-priced  contracts,  however  the 
overall,  service  set,  and  chapter  rating  assigned  a  high  rating  risk  to  this 
area. 

#20:  Motorola  SSG  is  assigned  a  high  risk  rating  for  performance  under 
product  quality.  It  is  highly  likely  there  will  be  a  major  impact  on 
hardware  performance  due  to  subcontractor  interface  specification 
requirements  for  continued  development.  “Subcontractor  Oversight”  is 
the  key  process  identified  to  manage  this  risk  and  no  risk  handling  tools 
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were  cited  due  to  the  delegated  nature  of  the  risk  management  area. 
Schedule  received  only  moderate  risk  ratings  because  the  situation  is  not 
expected  to  impact  current  build,  but  rather  those  in  the  future.  Cost  was 
rated  moderate  as  well,  even  though  this  is  a  fixed-price  effort,  due  to  the 
potential  for  out-of-scope  work  requirements.  No  specific  risk  handling 
tools  were  annotated,  although  it  was  noted  that  the  product  quality 
assurance  area  was  constantly  monitored  by  DCMA  and  any  changes 
would  be  promptly  noted  and  reflected  in  the  risk  ratings. 

#22:  McDonnell  Douglas  Helicopter  Systems  effort  for  the  Longbow 
Apache  program  was  assigned  a  high  risk  rating  in  all  three  areas  of 
performance,  schedule,  and  cost  for  product  quality.  Joint  Aircraft 
Inspection  was  the  first  identified  key  performance  parameter  and  was 
rated  as  high  risk  in  all  three  areas:  Performance  rating  indicated  that  a 
single  failure  could  result  in  loss  of  life  or  total  mission  failure  and 
product  technical  performance  requirements  continually  fail  acceptance 
criteria.  Schedule  rating  was  high  because  failure  to  repair  in  a  timely 
manner  would  likely  affect  the  remanufacturing  effort.  Cost  increase  was 
considered  likely.  “Product  Audits”:  100%  inspection  of  aircraft  when 
ready  for  inspection,  daily  was  the  identified  risk  handling  tool. 
“Maintenance”  was  the  second  performance  area  chosen  for  risk 
management  efforts  and  was  rated  as  moderate  risk  in  all  three  areas: 
Performance  rating  was  moderate  due  to  two  Corrective  Action  Reports 
(CARs)  being  issued  in  the  last  year,  not  all  mechanics  are  fully  trained, 
repetitive  errors,  and  aircraft  discrepancies  noted  during  customer 
inspection.  Schedule  risk  is  moderate  because  failure  to  detect 
deficiencies  during  this  process  that  incorporates  delivery  preparation  of 
aircraft,  would  impact  meeting  delivery  schedule.  High  cost  risk  due  to 
failure  to  detect  deficiencies  would  transfer  costs  of  correction  to  the 
customer.  “Product  Audits”  of  meeting  inspection  criteria  in  accordance 
with  aircraft  maintenance  publication  is  the  chosen  risk  handling  tool  and 
are  conducted  on  all  aircraft:  1 00%  intensity. 

#37:  Telechem  International  Inc.  received  high  risk  ratings  in  all  three 
areas  of  performance,  schedule,  and  cost  for  the  product  quality  area.  Five 
key  processes/systems  were  identified  for  risk  management  activity:  1. 
“CO A”  was  rated  as  high  risk  in  all  areas;  it  is  required  to  be  reviewed  and 
inspected  due  to  critical  application.  “Data  Analysis”  is  the  chosen  risk 
handling  tool  and  will  be  accomplished  with  meetings  with  the  contractor 
regarding  each  contract.  2.  “Contract  Review”  was  rated  as  high 
performance  and  cost  risk  and  moderate  schedule  risk.  Rationale 
indicated  it’s  critical  application  to  identify  requirements  and  no  further 
detail.  The  risk  handling  tool  is  a  “Contract  Award  Meeting”  for  each 
contract.  3.  “Inspection/Test”  shall  be  performed  due  to  product 
problems.  100%  “Product  Audits”  are  the  identified  risk  handling  tool 
using  Defense  Energy  Supply  Center  (DESC)  guidance.  4.  “Packaging 
and  Shipment”  received  high  risk  ratings  in  all  three  areas  due  to  high 
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failure  rate  upon  receipt  at  DESC.  “Product  Audits”  as  per  DESC 
guidelines  are  the  identified  risk  handling  tool.  5.  “Purchasing”  was 
assigned  a  high  performance  risk  rating  and  moderate  ratings  for  schedule 
and  cost.  These  ratings  were  assigned  due  to  questionable  documents  that 
were  hard  to  verify.  The  identified  risk  handling  tool  for  this  area  was 
“Process  Proofing/Product  Audit”. 

The  degree  of  possible  variation  for  key  processes/systems  is  so  great  for  this  area 
it  is  difficult  to  make  any  sort  of  meaningful  comparisons.  Although,  it  is  clear  that  the 
plans  are  not  carbon  copies  of  each  other  and  specifically  address  product  quality  issues 
for  the  contractual  effort  in  question  and  that  is  entirely  appropriate.  Product  Quality  for 
the  sampled  plans  addresses  risk  management  through  140  different  key  parameters,  over 
four  times  as  many  as  any  other  areas  in  the  RAMP  database.  With  such  a  strong 
presence  in  the  risk  management  data.  Product  Quality  is  a  key  driver  in  Product  Support 
service  set  level  risk  ratings  and  RAMP  plan  ratings  Overall.  Performance  risk  was  rated 
moderate  50%  of  the  time,  with  schedule  and  cost  each  ranked  as  low  53.3%  of  the  time. 
High  risk  ratings  were  assigned  to  these  areas  only  16.7%  and  13.3%  of  the  time, 
respectively. 

Some  commonalities  can  be  found  in  the  risk  handling  tools  used  to  mitigate 
Product  Quality  risk:  “Product  Audits”  and  some  version  of  “Test  and  Inspection”  are 
common  risk  mitigation  technique  applied  to  the  high  risk  Product  Quality  key 
processes/systems,  indicating  a  natural  Government  propensity  to  ensure  its  getting  what 
it  paid  for  prior  to  acceptance. 

9.  Packaging  Management  Program 

DCMA  provides  packaging  assistance  and  support  to  its  customers  to  ensure 

adequate  packaging  performance  in  accordance  with  the  contractual  arrangement  and  the 

item’s  physical  characteristics,  destination,  and  use.  DCMA  support  includes 
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surveillance  of  the  contractor’s  performance  and  capability  including  availability  of 
packaging  specification  information,  adequate  handling  processes,  equipment,  and 
packaging  costs.  The  goal  is  desired  protection  at  the  least  practical  cost  to  prevent 
deterioration  or  damage  until  customer  delivery.  Table  4.14  provides  an  overview  of  the 
key  processes/sy stems  chosen  for  the  packaging  management  program  under  the 
Packaging  Management  Program  One  Book  Chapter  4.4.4. 
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Table  4. 14.  Key  Processes/Systems  for  Packaging  Management  Program. 

(Source:  Developed  by  Researcher.) 


Seven  of  42  RAMP  plans  (16.7%)  addresses  the  packaging  management  program 
as  a  risk  management  area.  The  seven  plans  used  five  different  key  processes/systems  in 
two  different  combinations.  Clearly,  when  this  area  is  a  chosen  area  for  review,  there  is 
a  great  deal  of  continuity  between  the  plans.  This  is  likely  due  to  DCMA’s  policy  to 
maintain  a  Packaging  Management  Program  and  provide  Packaging  Specialists  to 
perform  functions  and  assist  in  the  packaging  process.  Such  specific  guidelines  easily 
lend  themselves  to  consistent  application  throughout  DCMA.  Three  key 
processes/system  were  used  in  all  plans:  “Handling”,  “Marking”,  and  “Packaging”. 
There  were  no  high  risk  areas  identified  under  the  packaging  management  program.  Five 
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plans  rank  all  three  risk  areas  as  low,  one  plan  ranks  risk  as  moderate,  and  one  remained 
“in  process”. 

E.  DELIVERY  RISK  RATING 

The  Delivery  service  set  employed  in  the  RAMP  database  corresponds  to  Chapter 
5  of  the  One  Book,  Delivery  Services  -  Right  Time.  Two  of  the  four  subchapters  are 
available  for  assigning  risk  ratings  in  RAMP:  Schedule  and  Delivery  Management  and 
Contract  Safety  Requirements.  Thirty-two  (32)  of  the  42  sampled  RAMP  plans  rated  risk 
areas  for  one  or  more  of  the  One  Book  Chapters  under  this  service  set.  The  following 
risk  management  plans — numbered  as  per  Appendix  A — assigned  risk  ratings:  1,4-9, 
13  -  19, 22  -  25,  and  29  -  42. 

The  individual  ratings  for  performance,  schedule,  and  cost  are  automatically 
generated  for  each  RAMP  plan  based  on  the  input  data  for  all  the  associated  One  Book 
Chapter  risk  ratings  for  each  of  these  areas.  Table  4.15  provides  an  overview  of  the 
service  set  risk  ratings  in  performance,  schedule,  and  cost  of  the  42  sampled  plans  from 
critical  and  strategic  suppliers.  Thirty-two  (32)  of  the  sampled  plan  addressed  risk 
management  under  the  Delivery  risk  area.  Ten  (10)  plans  rated  this  risk  as  not  applicable 
and  are  not  depicted  in  the  table. 


DELIVERY  RISK 

High 

Mod 

Low 

Performance 

7 

13 

12 

Schedule 

6 

11 

15 

Cost 

5 

9 

18 

Table  4.15.  Overview  of  the  Service  Set  Risk  Ratings  for  Delivery  Risk. 

(Source:  Developed  by  Researcher.) 


Delivery  is  the  second  most  applied  service  set  among  the  sampled  plans.  Thirty- 
two  plans  or  76.2%  of  the  RAMP  plans  ranked  risk  in  this  area.  While  the  majority  of  the 
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risk  assignments  in  this  area  were  like  elsewhere,  low  and  moderate,  this  service  set  had 
the  most  plans  rated  as  high  risk:  eight  plans  or  25%  of  the  plans  addressing  Delivery 
ranked  one  or  more  areas  of  performance,  schedule,  and  cost  as  high  risk. 

Of  the  two  assigned  One  Book  Chapters  for  risk  management,  Schedule  and 
Delivery  Management  was  applied  four  times  as  often  as  Contract  Safety  Requirements: 
69.0%  v.  16.7%  of  the  time. 

Eight  plans  rated  high  risk  at  the  Delivery  service  set  level  and  all  were  driven  by 
high  risk  ratings  under  Schedule  and  Delivery  Management.  The  sole  high  risk  rating  for 
Contract  Safety  Requirements  was  mitigated  at  the  Delivery  level  by  a  low  Schedule  and 
Delivery  Management  value  in  the  same  area.  When  applied,  Schedule  and  Delivery 
Management  seemed  to  take  a  more  prominent  role  in  the  risk  assignment  for  this  service 
set  as  a  whole. 

1.  Schedule  and  Delivery  Management 

DCMA’s  policy  is  to  improve  on-time  deliveries  by  reducing  delinquency  causes 
in  the  acquisition  process,  pre-notify  customers  of  potential  delays,  and  respond  to 
customer  inquiries.  These  activities  assist  the  customer  to  meet  readiness  requirements, 
identify  alternative  logistic  support  mechanisms,  and  select  proven  performers.  Table 
4.16  provides  an  overview  of  the  key  processes/system  chosen  for  schedule  and  delivery 
risk  management  under  the  Schedule  and  Delivery  Management  One  Book  Chapter  5.1. 
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Table  4. 1 6.  Key  Processes/System  for  Schedule  and  Delivery  Management. 
(Source:  Developed  by  Researcher.) 


Twenty-nine  (29)  of  42  RAMP  plans  (69.1%)  addressed  schedule  and  delivery  as 
a  risk  management  area.  The  29  plans  used  19  different  key  processes/systems  in  15 
different  combinations  to  assess  risk  for  the  contractor,  facility,  or  contractor  in  question. 
“Production  Planning  and  Control”  was  clearly  the  most  commonly  used  process,  used 
62.1%  of  the  time  RAMP  plans  assessed  the  contractors  schedule  and  delivery  system. 

Eight  RAMP  plans  rated  schedule  and  delivery  quality  risk  as  high.  The 
following  details  the  high  risk  rating  for  the  identified  areas  of  performance,  schedule, 
and  cost  and  the  associated  risk  handling  tools  chosen  to  mitigate  the  risk: 

•  #5:  Raytheon  Electronic  Systems  FI 8  Spare/Support  program  rated 

Schedule  and  Delivery  Management  a  high  risk  in  performance  and 
schedule  and  low  risk  in  cost.  “Production  Planning  and  Control”  was  the 
key  process/system  chosen  for  risk  management  efforts.  The  high  risk 
rating  for  performance  is  due  to  the  contractor’s  full  manufacturing 
capacity  that  is  unable  to  meet  delivery  schedules  required  by  the  contract. 
The  contractor  initially  lacked  an  Integrated  Master  Schedule  (IMS) 
between  three  disparate  facilities.  Now,  an  IMS  is  in  place  and  updated 
weekly.  Additionally,  subcontractors  have  been  added  to  assist  in  the 
production  effort.  Schedule  risk  is  high  due  to  missed  delivery  milestones 
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because  of  low  yields  on  components  and  a  shortage  of  Special  Test 
Equipment  for  the  increased  delivery  schedules.  Additionally,  employees 
are  required  to  increase  production.  Cost  risk  remains  low  due  to  die  firm 
fixed  price  contracts  in  place  for  the  program.  “Corrective  Action”  as 
needed  and  monthly  “Data  Analysis”  and  “Product  Audits”  are  the  tools 
identified  for  risk  handling  measures. 

#13:  Graco  Industries  is  given  a  high  risk  rating  in  all  three  areas  of 
performance,  schedule,  and  cost  for  schedule  and  delivery  management. 
“Production  Planning  and  Control”  is  the  key  process/system  chosen  for 
risk  management.  The  high  risk  performance  rating  is  due  to  the 
contractor’s  lack  of  a  well  managed  IMS  for  in-house  Government 
contracts,  a  high  turnover  of  personnel,  and  constant  lack  of  capacity. 
Schedule  is  rated  as  a  high  risk  because  the  contractor  is  on-time  less  than 
20%  of  the  time.  Even  though  this  is  a  fixed  price  contract,  DCMA 
justifies  its  high  cost  risk  rating  because  the  supplier  is  a  sole  source 
provider  and  there  are  “intangible  costs”  associated  with  failing  to  deliver 
on  time;  the  Government  lacks  other  options  should  they  fail  to  provide 
the  items  when  needed.  However,  Overall,  the  contractor  is  assigned  a 
low  cost  rating,  due  to  the  fixed  price  nature  of  the  contract.  Five  risk 
handling  tools  are  applied  to  this  problem:  1 .  “Alerts”  -  issued  each  time 
the  contractor  will  miss  the  final  delivery  date.  2.  “Contract  Abstract”  - 
complete  review  each  time  a  new  contract  or  change  order.  3. 
“Corrective  Action”  -  issue  Corrective  Action  Request  (CARs)  as 
necessary.  4.  “CPSS  Requests”  -  schedule  as  needed  when  received  from 
the  customer.  5.  “Production  Person  Workload  (PPW)  Report”  -  review 
the  PPW  for  past  due  orders  and  upcoming  orders  on  a  daily  basis 

#14:  Westinghouse  Electric  Corporation  facility  plan  rated  high  risk  in 
performance  and  schedule  and  moderate  cost  risk  for  schedule  and 
delivery.  “Production  Planning  and  Control”  is  the  area  chosen  for  risk 
management.  The  high  performance  rating  is  due  to  numerous  quality 
problems  associated  with  incomplete  data  packages  and  parts  not  within 
established  tolerances.  This  is  likely  caused  by  expedite  actions  leading  to 
circumvention  of  normal  lead  times.  Schedule  risk  is  rated  high:  the 
contractor  has  delivered  on-time  once  in  the  last  two  years  even  after 
receiving  many  contract  modifications  for  delivery  extensions.  Two 
CARs  have  been  issued  for  poor  schedule  trend  performance.  Cost  risk  is 
moderate  because  all  contracts  are  firm  fixed  price  with  no  progress 
payment;  however,  schedule  slippages  and  product  reworks  lead  to 
increased  cost  risk.  “Corrective  Action” — issuing  CARs  and  monthly 
“Schedule  Reviews” — 100%  delivery  schedules  for  all  contracts. 

#17:  Steward  &  Stevenson’s  facility  plan  was  rated  high  risk  in  all  three 
areas  of  performance,  schedule,  and  cost  for  schedule  and  delivery. 
“Production  Planning  and  Control”  is  the  area  chosen  for  risk  management 
because  it  is  the  top  level  system  that  controls  the  contractor’s  ability  to 
satisfy  the  delivery  schedule.  Performance  risk  is  rated  high  because  the 
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contractor  does  not  seem  to  have  a  well  managed  IMS  for  Government 
contracts.  Many  expedite  actions  are  required  and  vendor  control  is 
lacking.  The  contractor’s  less  than  50%  rate  for  on-time  delivery  drives 
the  high  schedule  risk  rating.  Lack  of  production  planning  results  in 
numerous  delayed  shipments.  Cost  risk  is  high  due  to  expediting  efforts. 
Five  risk  handling  tools  are  used  for  this  area:  1.  “Contract  Abstract”  - 
complete  review  each  time  a  new  contract  or  change  order.  2. 
“Corrective  Action”  -  issue  Corrective  Action  Request  (CARs)  as 
necessary.  3.  “CPSS  Requests”  -  schedule  as  needed  when  received  from 
the  customer.  4.  “Product  Audits”  -  per  shipment  each  time  the 
contractor  will  miss  the  final  delivery  date.  5.  “Production  Person 
Workload  (PPW)  Report”  -  review  the  PPW  for  past  due  orders  and 
upcoming  orders  on  a  daily  basis. 

#19:  Davies  Rail  &  Mechanical  assigned  high  risk  ratings  in  all  three 
areas  of  performance,  schedule,  and  cost  for  schedule  and  delivery 
management.  “Production  Planning  and  Control”  was  the  chosen  key 
process/system  for  risk  management.  The  contractor  did  not  have  a  well 
managed  IMS,  the  on-time  delivery  rate  is  less  than  50%,  and  expediting 
efforts  negatively  impact  the  cost.  Five  risk  handling  tools  were  chosen  to 
mitigate  the  risk:  1.  “Alerts”  -  each  time  contractor  will  miss  the  final 
delivery  date.  2.  “Contract  Abstract”  -  complete  review  each  time  a  new 
contract  or  change  order.  3.  “Corrective  Action”  -  issue  Corrective 
Action  Request  (CARs)  as  necessary.  4.  “CPSS  Requests”  -  schedule  as 
needed  when  received  from  the  customer.  5.  “Production  Person 
Workload  (PPW)  Report”  -  review  the  PPW  for  past  due  orders  and 
upcoming  orders  on  a  daily  basis. 

#22:  McDonnell  Douglas  Helicopter  Systems’  Longbow  Apache 
program  received  a  high  risk  rating  in  the  area  of  performance. 
“Forecasting”  and  “Production  Planning  and  Control”  are  the  identified 
key  process/systems  for  risk  management.  “Forecasting”  performance, 
schedule,  and  cost  were  all  rated  as  moderate  risks.  Performance  under 
“Production  Planning  and  Control”  was  rated  high:  High  turnover  of 
subcontractors  supplying  critical  and  flight  safety  parts  have  caused 
numerous  tooling  and  drawing  changes  and  increasing  probability  that 
performance,  schedule,  and  cost  objectives  will  not  be  met.  “Data 
Analysis”  is  the  risk  handling  tool  chosen  for  this  area. 

#41:  Raytheon  Tucson  ESSM  program  received  a  high  risk  ratings  for 
performance  and  schedule  and  a  low  risk  for  cost  under  Schedule  and 
Delivery  Management.  “Production  Planning  and  Control”  and  “Schedule 
and  Delivery  Management”  are  the  two  key  processes/systems  chosen  for 
risk  management.  Performance  risk  for  these  areas  stems  from  the 
contractor’s  rescheduling  of  major  programs  26  times  in  the  last  12 
months.  The  operational  Master  Performance  Schedule  (MPS)  schedule 
metric  fluctuates  between  70-80%;  internal  goals  have  never  been  met. 
The  schedule  rating  is  driven  by  the  additional  factor  of  only  a  66%  on- 
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time  delivery  rate.  Moderate  cost  rating  stems  from  the  contractor’s  use  of 
the  MRP  system  as  a  material  ordering  system  and  managing  by 
workarounds  or  expediting  which  negatively  impacts  cost.  Monthly  “Data 
Analysis”  and  “MMAS  Meetings  with  Contractor”  were  the  selected  risk 
handling  tools  for  “Production  Planning  and  Control”:  analyze  processes 
with  the  contractor  to  identify  root  causes  and  request  contractor  take 
corrective  action  on  this  system.  Risk  handling  tools  for  “Schedule  and 
Delivery  Management”  are  monthly  “Data  Analysis”  and  “Root  Cause 
Analysis”:  review  the  on-time  delivery  report  and  outstanding 

delinquency  report  to  identify  root  causes. 

#42:  Raytheon  Tucson  Evolved  Sea  Sparrow  Missile  (ESSM)  program 
rated  high  cost  risk  for  schedule  and  delivery  with  moderate  performance 
and  schedule  risk  assignments.  Two  key  processes/systems  were  chosen 
for  risk  review:  1.  “Product  Development”  assigned  cost  a  high  risk 
rating  due  to  cost  overruns  on  one  contract  and  obsolete  material  issues  on 
another  under  the  program.  2.  “Services  Management  Control  Process” 
assigned  a  high  risk  rating  to  cost  due  to  new  requirements  potentially 
causing  a  delayed  or  missed  milestone  which  may  result  in  costly 
rebaselining  activities.  The  chosen  risk  handling  tools  for  CESR  are  100% 
“Contract/Modification  Review”,  monthly  “Data  Analysis”,  weekly 
“Meetings”,  and  monthly  “Root  Cause  Analysis”. 


Seven  of  the  eight  plans  rating  high  risk  for  Schedule  and  Delivery  Management 

identified  “Production  Planning  and  Control”  as  a  key  process  to  manage  risk;  it  was  the 

sole  process  for  six  of  the  plans.  Plan  #42  is  the  exception  because  the  difficulties  do  not 

seem  to  stem  from  the  manufacturing  process  itself.  This  is  consistent  with  the  risk 

management  efforts  in  this  area  across  the  board  that  often  chose  this  process  as  a  key 

performance  parameter  for  risk  management.  A  common  theme  running  through  these 

plans  is  the  contractor’s  lack  of  an  integrated  master  scheduling  plan  between  facilities  or 

within  the  plant  to  distinguish  Government  contract  efforts  (#5,  #13,  #17,  #19,  and  #41). 

Other  plans  focus  on  various  subcontractor  difficulties  (#22),  quality  difficulties  (#14),  or 

obsolescence  and  new  requirements  issues  (#42).  In  all  cases,  the  high  risk  ratings  in  this 

area  drive  the  risk  rating  at  the  Delivery  service  set  level.  The  chosen  risk  handling  tools 

for  these  areas  seemed  common  within  the  specific  DCMA  office  responsible  for  contract 
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administration:  The  DCM  San  Antonio  office  tended  to  use  “Alerts”,  “Contract 
Abstracts”,  “CARs”,  “CPSS  Requests”,  and  “PPW  Report”  to  handle  risk.  Raytheon 
offices  tended  more  towards  “Data  Analysis”,  “Root  Cause  Analysis”,  and  “Meetings”. 

2.  Contract  Safety  Requirements 

When  contract  requirements  dictate  specific  safety  requirements  involving 
Ammunition  and  Explosives  (A&E),  Flight  Ground  Operations,  Industrial  Operations, 
Into-Plane  Refueling  Operations,  or  Maritime  Operations,  DCMA  will  evaluate 
contractor  high  risk  operations  in  accordance  with  the  DCMA  Contract  Safety  Program. 
Table  4.17  provides  an  overview  of  the  key  processes/systems  chosen  for  contract  safety 
requirements  under  Contract  Safety  Requirements  One  Book  Chapter  5.3. 
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REQUIREMENTS 
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Table  4. 1 7.  Key  Processes/Systems  for  Contract  Safety  Requirements. 
(Source:  Developed  by  Researcher.) 


Seven  of  42  RAMP  plans  (16.7%)  addressed  contract  safety  requirements  as  a 
risk  management  area.  The  six  plans  used  six  different  key  processes/systems  in  two 
configurations  to  assess  risk  for  the  contractor.  “Safety  Program”  was  the  system  used  in 
all  instances  and  clearly  drives  the  high  degree  of  consistency  in  this  area. 
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Only  one  RAMP  plan  rated  safety  risk  as  high.  The  following  details  the  high 
risk  rating  and  the  associated  risk  handling  methodology  to  mitigate  the  risk: 

•  #30:  Aerojet  General  Corporation  rates  performance  as  a  high  risk  area 

tinder  contract  safety  requirements.  “Safety  Program”  is  the  chosen  key 
process/system  to  manage  risk.  Performance  received  a  high  rating  due  to 
the  inherent  risk  of  explosives  handling  operations.  Any  failure  to  follow 
safety  requirements  would  significantly  increase  the  severity  of  mishaps. 
Although  no  specific  risk  handling  tool  was  listed,  risk  handling  detail 
indicated  a  Contract  Safety  Specialist  would  review  the  contractor’s  safety 
program  along  with  the  contractor’s  safety  representative  including 
operational  sites,  production  areas,  and  subcontractor  compliance. 


The  one  plan  with  a  high  risk  rating  used  the  same  key  system  to  manage  risk  as 

the  other  plans  under  Contract  Safety  Requirements:  “Safety  Program”,  selected  because 

it  is  the  overarching  key  process  under  this  area.  The  rating  rationale  is  sound,  based  on 

the  potential  impact  vice  the  probability  of  the  risk  event.  For  this  reason,  it  is  reasonable 

that  a  low  risk  rating  for  Schedule  and  Delivery  Management  mitigates  the  high  risk 

rating  here  for  the  service  set.  While  not  specifically  spelled  out,  risk  handling  will  be 

accomplished  by  reviewing  the  contractor’s  safety  program 

F.  BUSINESS  AND  FINANCIAL  SYSTEMS  RISK  RATING 

The  Business  and  Financial  Systems  service  set  employed  in  the  RAMP  database 

corresponds  to  Chapter  7  of  the  One  Book,  Business  &  Financial  Systems  Services.  Four 

of  the  six  subchapters  are  available  for  assigning  risk  ratings  in  RAMP:  Contract 

Property  Management,  Contractor  Estimating  System  Reviews,  Material  Management 

and  Accounting  Systems,  and  Cost  Accounting  Standards  (CAS)  Administration. 

Nineteen  (19)  of  the  42  sampled  RAMP  plans  rated  risk  areas  for  one  or  more  of  the  One 

Book  Chapters  under  this  service  set.  The  following  risk  management  plans — numbered 


82 


as  per  Appendix  A — assigned  risk  ratings:  1,  8,  10,  18,  22,  25,  27,  28,  30,  31,  33  -  36, 
and  38  -  42. 

The  individual  ratings  for  performance,  schedule,  and  cost  are  automatically 
generated  for  each  RAMP  plan  based  on  the  input  data  for  all  the  associated  One  Book 
Chapter  risk  ratings  for  each  of  these  areas.  Table  4.18  provides  an  overview  of  the 
service  set  risk  ratings  in  performance,  schedule,  and  cost  of  the  42  sampled  plans  from 
critical  and  strategic  suppliers.  Nineteen  (19)  of  the  sampled  plan  addressed  risk 
management  under  the  Business  and  financial  Systems  risk  area.  Twenty-four  (24)  plans 
rated  this  risk  as  not  applicable  and  are  not  depicted  in  the  table. 

Business  and  Financial  Systems  are  used  for  risk  management  efforts  in  45.2%  of 
the  sampled  plans.  The  majority  of  the  risk  was  rated  low:  63.2%  of  plans  rated 
schedule  and  cost  risk  as  low,  while  47.4%  (still  the  largest  proportion)  rated 
performance  risk  as  low.  High  risk  ratings  were  rare:  twice  for  performance  and  cost 
and  one  for  schedule.  This  is  undoubtedly  indicative  of  the  fixed  price  contract  types 
used  commonly  throughout  the  plans. 


BUSINESS  & 

FINANCIAL  SYSTEMS 

RISK 

High 

Mod 

Low 

Performance 

2 

8 

9 

Schedule 

1 

6 

12 

Cost 

2 

5 

12 

Table  4. 1 8.  Overview  of  the  Service  Set  Risk  Ratings  for  Business  and  Financial 

Systems  Risk. 

(Source:  Developed  by  Researcher.) 

Risk  ratings  were  fairly  well  dispersed  among  the  four  One  Book  Chapters  used 

for  the  Business  and  Financial  Systems  service  set  with  over  half  the  plans  using  this 
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service  set  assigning  risk  ratings  for  Contractor  Estimating  System  Reviews  (CESRs) 
and  Material  Management  and  Accounting  Systems  (MMAS)  (63.2%),  Contract  Property 
Management  (73.7%),  and  Contractor  Purchasing  System  Reviews  (73.7%). 

Only  three  plans  (#25,  #41,  and  #42)  rated  high  risk  at  the  Business  and  Financial 
service  set  level  and  all  three  were  driven  by  high  risk  ratings  for  Contract  Property 
Management  which  seems  to  be  the  most  risky  area  and  a  key  driver  in  this  service  set. 
CESRs  provided  the  other  sole  high  risk  rating  and  contributed  to  high  risk  service  set 
rating  (#41). 

1.  Contractor  Estimating  System  Reviews 

Contractor  Estimating  System  Reviews  (CESRs)  review  the  contractor’s 
processes  of  collecting  and  building  cost  estimates.  The  Government  must  ensure  this  is 
done  according  to  standards  with  the  right  information  source  and  the  right  system  to 
produce  reliable  and  consistent  cost  information  representative  of  actual  costs.  Table 
4.19  provides  an  overview  of  the  key  processes/systems  chosen  for  CESR  risk 
management  under  the  Contractor  Estimating  System  Reviews  One  Book  Chapter  7.3. 

Twelve  (12)  of  42  RAMP  (28.6%)  plans  address  CESRs  as  a  risk  management 
area.  The  12  plans  use  nine  different  key  processes/systems  in  eight  different 
combinations  to  assess  risk  for  the  contractor,  facility,  or  contract  in  question.  “Forward 
Pricing”  was  the  most  often  used  process,  identified  as  key  area  66.7%  of  the  time,  with 
“Cost  Accounting  System  (CAS)”  and  “Proposal  Development”  following  closely,  in  use 
58.3%  of  the  time.  Combined,  these  clearly  lend  some  continuity  to  the  risk  management 
process. 
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CONTRACTOR 

ESTIMATING  SYSTEM 

REVIEW 

RISK  MANAGEMENT 

PLANS  NO.s 

EY  PROCESSES/SYSTEMS  1  10  22  28  30  31  33  34  35  38  39  41 

ccounting 

X 

ost  Accounting  System  (CAS) 

X 

X 

X 

X 

X 

X 

X 

stimating  System 

X 

orward  Pricing 

X 

X 

X 

X 

X 

X 

X 

X 

MAS 

X 

X 

X 

egotiate  Final  Overhead  Rates 

X 

roposal  Development 

X 

X 

X 

X 

X 

X 

X 

urchasing  System 

X 

X 

ystem  Audit 

X 

X 

_ 

_ 

1 

Table  4.19.  Key  Processes/Systems  for  Contractor  Estimating  Systems  Review. 

(Source:  Developed  by  Researcher.) 

Only  one  RAMP  plan  rated  CESRs  as  a  high  risk  area: 

•  #41 :  Raytheon  Tucson  Systems  plan  was  rated  as  a  high  risk  in  the  area 

of  cost  for  CESRs  with  performance  and  schedule  being  assigned 
moderate  ratings.  Three  key  processes/systems  were  chosen  for  risk 
management:  “Cost  Accounting  System”,  “Forward  Pricing”,  and 

“Proposal  Development”.  All  three  areas  were  individually  rated  as 
moderate  and  do  not  support  the  high  cost  risk  rating  at  the  One  Book 
Chapter  level.  The  chapter  narrative  indicates  a  corrective  action  in  place 
for  inadequate  and  late  subcontract  cost/price  analysis. 


The  lack  of  supporting  ratings  at  the  key  process/system  level  is  inconsistent  with 
the  scheme  of  risk  management  in  the  RAMP  system  which  is  designed  to  build  up  from 
the  lowest  levels  of  key  processes/systems  through  to  an  Overall  rating  supported  and 
documented  by  the  ratings  at  the  lower  echelons.  Additionally,  the  lack  of  supporting 
information  at  these  levels  means  there  is  no  direct  correlation  between  the  three  chosen 
processes/systems  (and  their  associated  risk  handling  tools)  and  problem  at  hand. 
However,  high  risk  under  Contract  Property  Management  lends  additional  credibility  to 
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the  resulting  high  risk  at  the  Business  and  Financial  Systems  service  set  level.  The  three 
chosen  systems  for  risk  management  are  the  three  most  commonly  used  areas  for  risk 
assessment  under  CESR. 

2.  Material  Management  and  Accounting  Systems 

The  concept  of  Material  Management  and  Accounting  Systems  (MMAS)  is  to 

ensure  that  material  used  to  manufacture  a  product  is  charged  or  costed,  in  the  right 
amount,  to  the  contract  for  that  product  and  no  other.  Suppliers  may  have  numerous 
contractors,  both  Government  and  commercial  and  properly  assigning  material  costs  to  a 
contract  can  be  a  complex  and  confusing  enterprise.  DCMA  must  apply  risk 
management  to  ensure  confidence  in  allowable  and  allocable  material  costs  assignable  to 
a  contract.  Table  4.20  provides  an  overview  of  the  key  processes/systems  chosen  for 
MMAS  risk  management  under  the  Material  Management  and  Accounting  Systems  One 
Book  Chapter  7.5. 

Twelve  (12)  of  42  RAMP  plans  (28.6%)  addressed  MMAS  as  a  risk  management 
area.  The  12  plans  used  15  different  key  processes/systems  in  10  different  configurations 
to  assess  risk  for  the  contractor.  “Accounting  System  Reviews”  was  the  system  used 
most  commonly,  66.7%  of  the  time.  No  RAMP  plans  rated  risk  as  high  in  this  area  and 
the  predominant  risk  ratings  were  low  for  performance,  schedule,  and  cost:  75%,  91 .6%, 
and  83.3%  of  the  plans  addressing  MMAS. 
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MATERIAL  MANAGEMENT  & 
ACCOUNTING  SYSTEM  (MMAS) 

RISK  MANAGEMENT 
PLANS  NO.s 

1  22  28  30  31  33  34  35  36  38  40  41 
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H 
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Table  4.20.  Key  Processes/Systems  for  Material  Management  and  Accounting  System 

(MMAS). 

(Source:  Developed  by  Researcher.) 

3.  Contract  Property  Management 

Contractors  must  have  an  adequate  system  to  manage  Government  property  in 
their  possession.  Their  property  control  system  must  serve  to  control,  protect,  preserve, 
maintain  and  establish  accountability  over  Government  property.  DCMA  oversight 
includes  activities  to  assess  the  contractor’s  system  to  determine  priority,  degree,  and 
level  of  surveillance  required;  validate  a  contractor’s  self  oversight  program;  perform 
property  administration  functions;  and  investigate  loss,  damage,  or  destruction  (LDD)  of 
Government  property.  Table  4.21  provides  an  overview  of  the  key  processes/sy stems 
chosen  for  risk  management  under  the  Contract  Property  Management  One  Book  Chapter 
7.1. 
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Fourteen  (14)  of  42  RAMP  plans  (33.3%)  addressed  contract  property 
management  as  a  risk  management  area.  The  14  plans  used  16  different  key 
processes/systems  in  five  configurations  to  assess  contractor  risk.  Eight  of  the  plans 
(57.1%)  used  the  same  configuration  and  12  plans  used  Property  Management  as  a  key 
processes  (85.7%),  providing  a  significant  amount  of  continuity  between  the  plans.  Even 
among  five  different  combinations  there  is  significant  congruence/overlap  between  the 
chosen  processes/systems. 

Five  RAMP  plans  rated  property  management  as  a  high  risk  area.  The  following 
details  the  high  risk  rating  for  the  identified  area  of  performance,  schedule,  and  cost  and 
the  associated  risk  handling  tools  chosen  to  mitigate  the  risk: 


CONTRACT  PROPERTY  ~ 
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Table  4.2 1 .  Key  Processes/Systems  Chosen  for  Contract  Property  Management. 

(Source:  Developed  by  Researcher.) 
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#10:  Brown  &  Root  Services  Corporation  facility  is  rated  as  high  risk  in 
all  three  areas  of  performance,  schedule,  and  cost.  Fifteen  different  key 
processes/systems  are  used  to  manage  risk,  seven  of  these  areas  are  rated 
as  high  risk:  “Acquisition”,  “Contractor  Property  Close-out”, 

“Disposition”,  “Property  Management”,  “Records”,  “Reports”,  and 
“Subcontractor  Control”.  Performance  risk  is  high  due  to  the  variety  and 
geographic  dispersion  of  Government  property  under  the  contractor’s 
cognizance:  over  95,690  line  items  scattered  throughout  the  Balkans 
(Bosnia,  Croatia,  Hungary,  Macedonia,  and  Kosovo)  and  valued  at  over 
$293,046,299.  Schedule  risk  is  high  due  to  potential  impact  of  systemic 
deficiencies  on  the  contractor’s  ability  to  order  materials  and  issue 
subcontracts  to  meet  the  numerous  and  varied  requirements.  Cost  risk  is 
high  due  to  the  cost  reimbursable  contracts  and  performance  in  multiple 
locations.  System  Evaluation  is  the  chosen  risk  handling  tool  to  mitigate 
risk  for  this  contractor:  an  annual  property  control  system  audit. 

#25 :  DRS  Infrared  Technologies  LP  Ml  A2  Abrams  Upgrade  program  is 
assigned  a  high  risk  rating  in  the  area  of  performance  with  low  risk  ratings 
for  schedule  and  cost.  The  contractor’s  performance  criteria  are  divided 
into  three  sub-elements:  inherent,  property  control  system,  and  property 
control  system  changes.  Property  control  system  and  property  control 
system  changes  are  assigned  a  high  rating  because  the  contractor  is  new 
and  acceptable  property  control  procedures  have  not  been  submitted  and 
potential  changes  are  unknown.  There  are  no  known  deficiencies  now  that 
could  impact  cost  or  schedule.  “Property  Management”  is  the  chosen  key 
system  for  risk  management.  “System  Evaluation”  using  annual  sampling 
is  the  chosen  risk  handling  tool. 

#40:  Raytheon  Tucson  AMRAAM  program  is  assigned  a  high  risk 
rating  for  performance,  schedule,  and  cost  for  contractor  property 
management.  This  risk  rating  is  not  broken  down  into  the  specific 
elements  but  cites  excessive  Lost,  Damaged,  and  Destroyed  (LDD) 
property  on  the  program  as  rationale  for  the  “Property  Management”  key 
process.  “System  Evaluation”  using  an  annual  sample  is  the  chosen  risk 
handling  tool. 

#41:  Raytheon  Tucson  Systems  is  assigned  high  risk  ratings  for 
performance,  schedule,  and  cost.  Fifteen  different  key  processes/systems 
are  used  to  manage  risk,  six  of  these  areas  are  rated  as  high  risk: 
“Acquisition”,  “Movement”,  “Property  Management”,  “Records”, 
“Subcontractor  Control”,  and  “Utilization”.  The  contractor’s  property 
system  is  rated  as  “unsatisfactory”  but  “approved”  and  the  contractor  is 
pursuing  an  approved  corrective  action  plan  (CAP)  and  joint  audits  with 
DCMA  personnel  to  improve  their  “internal”  ratings  from  “RED”  to 
“YELLOW”.  “Annual  Statistical  Sampling”  is  the  chosen  risk  handling 
tool  in  all  instances. 
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•  #42:  Raytheon  Tucson  Evolved  Sea  Sparrow  Missile  (ESSM)  was 

assigned  high  risk  ratings  in  all  three  areas  of  performance,  schedule,  and 
cost  for  property  management.  “Property  Management”  is  the  chosen  key 
system  for  risk  management:  Performance  is  rated  high  due  to  the 
contractor’s  unsatisfactory  performance  during  a  property  control  system 
audit.  Process  integrity  is  compromised  due  to  improper  management  and 
control  of  Government  property  and  may  impact  schedule.  Problems  with 
the  system  cause  overall  cost  increases.  An  annual  systems  analysis  using 
100%  “Judgement  Sampling”  is  the  chosen  risk  handling  tool. 

The  five  plans  rating  high  risk  for  Contract  Property  Management  all  had  a 
common  key  process/system  in  common:  “Property  Management”.  Two  of  the  plans 
(#10  and  #41)  used  15  different  key  processes/systems  to  assess  risk  for  the  contractor; 
three  of  the  plans  (#25,  #40,  and  #42)  used  only  one  key  parameter  for  this  purpose.  Four 
of  the  plans  (#10,  #40,  #41,  and  #42)  rated  all  three  areas  of  risk,  performance,  schedule, 
and  cost  as  high  risk,  while  one  plan  (#25)  rated  only  performance  as  a  high  risk, 
schedule  and  cost  risk  remain  low.  However,  despite  the  preponderance  of  high  risk 
ratings  in  this  area,  these  factors,  when  encompassed  with  the  ratings  for  the  other  three 
One  Book  Chapter  level  ratings,  contributed  to  moderate  risk  ratings  and  only  led  to  three 
plans  with  high  risk  ratings  at  the  service  set  level.  “System  Evaluation”  was  the  chosen 
risk  management  tool  in  three  instances  (#10,  #25,  and  #40),  with  “sampling”  tools  used 
in  the  remaining  two  plans  (#41  and  #42).  However,  “System  Evaluation”  is  conducted 
using  annual  sampling,  which  blurs  the  distinction  between  the  two  tools. 

4.  Contractor  Purchasing  System  Reviews 

Contractor  Purchasing  System  Reviews  (CPSRs)  involve  the  Administrative 
Contracting  Officer’s  (ACO’s)  consent  for  the  prime  to  place  subcontracts  and  approval 
of  the  contractor’s  purchasing  system.  When  the  prime  contractor  awards  subcontracts 
non-competitively  or  the  contract  allows  all  subcontract  costs  to  flow  up  to  the 
Government,  the  Government  is  placed  at  risk.  Purchasing  system  approval  allows 
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contracting  officers  to  waive  subcontract  advance  notifications  and/or  consent  to 
subcontract  actions  and  provide  early  CAS  information  to  base  source  selection  decisions 
and  negotiation  positions  for  profit/fee.  Table  4.22  provides  an  overview  of  the  key 
processes/system  chosen  for  risk  management  under  Consent  to  Subcontract/Contractor 
Purchasing  Review  One  Book  Chapter  7.4. 

Nine  of  42  RAMP  plans  addressed  contractor  purchasing  systems  as  a  risk 
management.  The  nine  plans  used  11  different  key  processes/systems  in  eight  different 
configurations  to  assess  risk  for  the  contractor.  “Best  Value”  and  “Internal  Purchasing 
System  Audit”  were  the  two  most  commonly  used  systems/processes  for  risk 
management:  55.6%.  No  RAMP  plans  rated  high  risk  in  this  area.  The  majority  of  the 
risk  ratings  were  low:  66.7%  for  schedule  and  55.5%  each  for  performance  and  cost. 


CONTRACTOR 

PURCHASING  SYSTEM 

REVIEW 

RISK  MANAGEMENT 
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Internal  Purchasing  System 
Audit 

X 

X 
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X 
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X 
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Price  Negotiation 

X 

X 

Public  Law 

X 

X 

Purchasing/Contract 

X 

Summary  of  Processes 

X 

System  Approval 

X 

X 

Vendor  Rating 

X 

X 

X 

X 

Table  4.22.  Key  Processes/System  for  Contractor  Purchasing  System  Review. 
(Source:  Developed  by  Researcher.) 
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G.  PAYMENT  AND  FINANCIAL  MANAGEMENT  RISK  RATING 

The  Payment  and  Financial  Management  service  set  employed  in  the  RAMP 

database  corresponds  to  Chapter  9  of  the  One  Book,  Payment  and  Financial  Management 
Services.  Three  of  the  six  subchapters  are  available  for  assigning  risk  ratings  in  RAMP: 
Progress  Payments,  Performance  Based  Payments,  and  Public  Vouchers.  Twenty-one 
(21)  of  the  42  sampled  RAMP  plans  rated  risk  areas  for  one  or  more  of  the  One  Book 
Chapters  under  this  service  set.  The  following  risk  management  plans — numbered  as  per 
Appendix  A — assigned  risk  ratings:  1 , 4  -  6, 19, 22  -  25, 28, 30  -  35,  and  38  -  42. 

The  individual  ratings  for  performance,  schedule,  and  cost  are  automatically 
generated  for  each  RAMP  plan  based  on  the  input  data  for  all  the  associated  One  Book 
Chapter  risk  ratings  for  each  of  these  areas.  Table  4.23  provides  an  overview  of  the 
service  set  risk  ratings  in  performance,  schedule,  and  cost  of  the  42  sampled  plans  from 
critical  and  strategic  suppliers.  Twenty-one  (21)  of  the  sampled  plans  addressed  risk 
management  under  the  Payment  and  Financial  Management  risk  area.  Twenty-one  (21) 
plans  rated  this  risk  as  not  applicable  and  are  not  depicted  in  the  table. 

Payment  and  Financial  Management  was  applied  by  half  of  the  sampled  RAMP 
plans.  An  absolute  majority  of  the  risk  at  the  service  set  level  was  rated  low:  76.2%  for 
performance  and  schedule  and  71.4%  for  cost.  Only  one  plan  (#23)  ranked  high  risk  at 
the  service  set  level. 
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PAYMENT  & 

FINANCIAL 
MANAGEMENT  RISK 

High 

Mod 

Low 

Performance 

1 

4 

16 

Schedule 

1 

4 

16 

Cost 

1 

5 

15 

Table  4.23.  Overview  of  the  Service  Set  Risk  Ratings  for  Payment  and  Financial 

Management  Risk. 

(Source:  Developed  by  Researcher.) 

Of  the  three  assigned  One  Book  Chapters  for  risk  management,  Progress 
Payments  Based  on  Costs  and  Public  Voucher  were  applied  most  often  (61.9%  and 
76.2%  respectively),  while  Performance  Based  Payments  was  used  less  than  half  the  time 
risk  was  rated,  42.9%. 

Only  two  plans  used  any  high  risk  ratings  and  these  were  in  the  Performance 
Based  Payments  area  (#23)  which  ultimately  drove  high  risk  ratings  at  the  service  set 
level  and  in  the  Public  Vouchers  area  (#42)  which  was  mitigated  to  moderate  risk  ratings 
at  the  service  set  level  by  low  and  moderate  ratings  in  the  other  two  One  Book  Chapters. 

1.  Progress  Payments  Based  on  Cost 

Progress  payments  recognize  a  contractor’s  need  for  working  capital  due  to  long 
lead  times  and  work  in  process  costs  and  thus  provide  interim  financing  for  contracts 
other  than  cost-reimbursement  arrangements.  DCMA’s  role  regarding  the  management 
of  progress  payments  is  three-fold:  ensure  that  Government  funds  are  protected,  that  the 
contractor  is  paid  in  a  timely  fashion  commensurate  with  the  actual  work  performed  as 
per  contractual  requirements,  and  that  overpayments  are  avoided.  To  do  this  the 
contractor’s  management  systems,  financial  condition,  and  contract  performance  must  be 
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monitored.  Table  4.24  provides  an  overview  of  the  key  processes/systems  chosen  for  risk 
management  under  the  Progress  Payments  One  Book  Chapter  9.2. 

Thirteen  (13)  of  42  RAMP  plans  (31.0%)  addressed  progress  payments  as  a  risk 
management  area.  The  13  plans  used  five  different  processes/systems  in  seven  different 
configurations  to  assess  risk  for  the  contractor.  “Management  of  Company  Financial 
Condition”  was  the  most  frequently  used  process,  84.6%  of  the  time.  Clearly  there  is  a 
lot  of  continuity  between  the  various  plans,  with  a  majority  of  the  plans  using  the  exact 
same  configuration.  No  one  plan  used  a  key  process  or  system  not  used  elsewhere.  No 
RAMP  plans  rated  high  risk  in  this  area.  An  absolute  majority  of  risk  ratings  were  low: 
84.6%  for  performance,  92.3%  for  schedule,  and  76.9%  for  cost. 


PROGRESS  PAYMENTS  “ 

RISK  MANAGEMENT 

PLANS  NO.s 

KEY  PROCESSES/SYSTEMS  1  4  5  22  25  28  30  31  32  33  38  41  42 

1 

r 

r 

r 

n 

n 

Management  of  Company 
Financial  Condition 

H 

H 

H 

X 

H 

l 

X 

H 

i 

X 

X 

Management  of  Costs 

□ 

□ 

Q 

Q 

■ 

■ 

□ 

B 

□ 

1 

■ 

□ 

IB 

Management  of  Business 
Systems 

X 

X 

X 

X 

1 

l 

i 

X 

X 

X 

1 

X 

Management  of  Production  & 
Quality  Assurance  (QA) 

X 

X 

X 

X 

1 

l 

i 

1 

X 

X 

■ 

1 

X 

Management  of  Progress 
Payment  Requests,  Preparation, 
&  Submittal 

X 

X 

X 

X 

1 

X 

X 

i 

1 

X 

Table  4.24.  Key  Processes/Systems  for  Progress  Payments. 

(Source:  Developed  by  Researcher.) 

2.  Public  Vouchers 

Contractors  submit  interim  and  final  public  vouchers  for  costs  and  fees  under 
cost-reimbursement,  time-and-materials  (T&M),  and  labor-hour  (LH)  contracts.  DCMA 
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contract  auditors  are  authorized  representatives  of  the  Administrative  Contracting  Officer 
(ACO)  for  receiving  vouchers,  approving  interim  vouchers,  authorizing  contractor  direct 
submission  to  the  disbursing  office  for  those  suppliers  with  approved  billing  systems,  and 
forwarding  final  payment  vouchers  to  the  ACO  for  approval.  The  auditor  may  be  the 
Defense  Contract  Audit  Agency  (DCAA).  Table  4.25  provides  an  overview  of  the  key 
processes/systems  chosen  for  risk  management  under  the  Public  Vouchers  One  Book 
Chapter  9.4. 

Seventeen  (17)  of  42  RAMP  plans  (40.5%)  addressed  public  vouchers  as  a  risk 
management  area.  The  17  plans  used  six  different  key  processes/systems  in  ten  different 
combinations  to  assess  risk  for  the  contractor.  “Management  of  Voucher  Preparation  and 
Submittal”  was  the  most  commonly  chosen  area  for  risk  management,  used  82.4%  of  the 
time.  One  again  there  is  visibly  a  great  deal  of  continuity  between  the  plans  and  their 
chosen  methodologies.  A  good  deal  of  overlap  exists  even  when  there  is  variation. 


PUBLIC  VOUCHERS 

RISK  MANAGEMENT 

PLANS  NO.s 

KEY 

PROCESSES/SYSTEMS  I  4  6  22  23  24  25  28  30  32  33  34  35  39  40  41  42 

_ 

Accounting  System 

Contractor  Procedures 

X 
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■ 
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X 

X 

X 

1 

1 

1 

X 

X 

X 

X 

X 

X 

X 

1 

Management  of  Business 
Systems 

X 

X 

X 

X 

1 

1 

1 

1 

X 

X 

X 

X 

Management  of  Costs 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

B 

B 

B 

B 

Management  of  Voucher 
Preparation/Submittal 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

Table  4.25.  Key  Processes/Systems  for  Public  Vouchers. 
(Source:  Developed  by  Researcher.) 
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Only  one  RAMP  plan  rated  risk  as  high  in  this  area: 

•  #42:  Raytheon  Tucson  ESSM  program  was  rated  as  high  risk  in  the 

areas  of  schedule  and  cost ;  performance  risk  was  rated  moderate.  Four 
key  processes/systems  are  chosen  to  manage  risk,  as  indicated  above. 
Only  the  “Management  of  Costs”  process  received  high  risk  ratings:  This 
area  was  chosen  because  only  allocable,  reasonable  costs  are  allowable. 
Rating  rationale  indicated  that  the  contractor  had  notified  the  Government 
that  additional  funds  are  needed;  past  performance  indicates  previous  cost 
control  problems;  and  Limitation  of  Cost/Limitation  of  Funds  notifications 
were  not  being  provided  to  the  Government  until  requested  and  then,  late. 
Performance  progress  consistently  lags  funding.  The  chosen  risk  handling 
tool  for  this  area  is  100%  monthly  “Audit  Voucher  for  Fee”. 

The  key  processes/systems  chosen  for  risk  management  were  consistent  and  used 
extensively  by  other  plans  addressing  risk  management  under  the  Payment  and  Financial 
Management  service  set.  But  only  one  of  the  three  rated  risk  as  high:  “Management  of 
Costs”.  The  rationale  for  this  clearly  justified  the  risk  rating  and  its  importance.  The 
high  risk  rating  here  however,  did  not  produce  a  high  risk  rating  at  the  service  set  level 
due  to  the  mitigation  effect  of  other  One  Book  Chapter  risk  ratings  for  the  plan.  The 
chosen  risk  handling  plan,  “Audit  Voucher  for  Fee”  seems  consistent  with  the  need  to 
verify  costs. 


3.  Performance  Based  Payments 

Performance  based  payments  provide  contractor  financing  vice  payment  for 
accepted  items.  It  is  applicable  when  objective  and  quantifiable  performance 
measurements  exist  or  when  completion  of  definable  events  is  appropriate.  DCMA  is 
responsible  for  administrating  payments  under  this  program  which  is  preferred  due  to  the 
clearly,  definable  links  between  performance  and  dollars  and  allowance  for  the 
establishment  of  clear  goals.  Performance  Based  Payments  are  used  for  fixed  price  type 
contracts  when  no  other  financing  is  provided  for.  Table  4.26  provides  an  overview  of 
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the  key  processes/systems  chosen  for  risk  management  under  the  Performance  Based 
Payments  One  Book  Chapter  9.3. 

Nine  of  42  RAMP  plans  (21.4%)  addressed  Performance  Based  Payments  as  a 
risk  management  area.  The  nine  plans  used  seven  different  key  processes/systems  in 
seven  different  configurations  to  assess  risk  for  the  contractor.  “Management  of 
Performance  Based  Requirements,  Preparation,  and  Submittal”  was  by  far  the  most 
commonly  used  process,  occurring  77.8%  of  the  time.  RAMP  plan  #34  was  in  process 
of  rating  risk  for  this  area  and  provided  no  key  process/system  information. 


PERFORMANCE  BASED  PAYMENTS 

RISK  MANAGEMENT 
PLANS  NO.s 

5  19  22  23  34  39  40  41  42 

KEY  PROCESSES/SYSTEMS 

Accomplishment  of  Performance 
Certification 

X 

Completion  of  Contract  Milestones 

X 

X 

X 

Management  of  Business  Systems 

X 

X 

Management  of  Company  Financial 
Condition 

X 

X 

X 

X 

X 

Management  of  Costs 

X 

X 

X 

X 

Management  of  Production/QA  &  Physical 
Percent  of  Completion 

X 

X 

Management  of  Performance  Based 
Requirements,  Preparation,  &  Submittal 

X 

X 

X 

X 

X 

X 

X 

Table  4.26.  Key  Processes/Systems  for  Performance  Based  Payments. 

(Source:  Developed  by  Researcher.) 

Only  one  RAMP  plan  rated  risk  as  high  for  performance  based  payments: 

•  #23:  Lockheed  Martin  Missiles  and  Fire  Control  was  rated  as  high  risk 

in  all  three  areas  of  performance,  schedule,  and  cost.  “Management  of 
Performance  Based  Payment  Requirements,  Preparation,  and  Submittal”  is 
the  chosen  risk  management  area.  Physical  verification  reviews  indicated 
delinquent  subcontractors.  Production  lines  are  co-mingled  with  other 
contract  efforts.  Contractor  is  financially  stable,  however  a  potential 
merger  may  affect  this.  A  moderate  number  of  changes  is  anticipated  to 
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require  renegotiation  of  events  and  anticipated  major  changes  may  expose 
the  contract  to  greater  risk  than  previously  known.  The  chosen  risk 
handling  tools  for  this  process  are  100%  monthly  “Data  Analysis”  100% 
annual  “DCAA  Audit”  of  incurred  costs  to  verify  performance  payments 
are  not  advance  payments,  initially  “Established  Surveillance  Plan”,  100% 
“Product  Audits”  as  submitted  for  approval,  and  monthly  100%  “Review 
of  Paid  Vouchers”. 

The  high  risk  plan  uses  only  one  key  parameter  to  manage  risk  for  the  contractor 
under  Payment  and  Financial  Management:  “Management  of  Performance  Based 
Requirements,  Preparation,  &  Submittal.”  This  system  is  commonly  used  in  the  other 
plans  under  this  service  set,  but  usually  in  conjunction  with  other  key  parameters.  The 
high  risk  ratings  here  drive  the  high  risk  ratings  at  the  service  set  level.  The  rationale 
noted  several  problem  areas  to  defend  the  high  risk  rating.  Five  different  risk  handling 
tools  are  listed  and  would  seem  to  encompass  the  problem  through  their  combined 
umbrella  effect  regarding  nearly  every  aspect  of  this  area. 

H.  VARIABILITY  IN  THE  SAMPLED  PLANS 

The  following  sections  address  a  few  summary  observations  regarding  variability 

in  the  RAMP  plans  as  a  whole: 

1.  Missing  Risk  Ratings 

The  risk  rating  process  for  the  RAMP  plans  start  with  risk  planning.  Planning 
involves  the  identification  of  key  processes/systems  that  can  have  a  significant  adverse 
affect  on  performance,  schedule,  or  cost  if  not  properly  controlled.  The  “significant 
adverse  affect”  is  defined  based  on  probability  of  occurrence  and  impact.  Those 
processes  or  systems  meeting  this  requirement  as  determined  by  DCMA  personnel  are 
rated  in  the  RAMP  plan.  Contract  requirements,  memorandums  of  agreement  (MOAs) 
with  the  customer,  or  other  delegations  such  as  subcontract  work  provide  additional 
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reasons  to  rate  specific  processes  or  systems  not  otherwise  identified  as  a  ratable  item  by 
DCMA. 

The  RAMP  plans  typically  indicate  why  a  process  or  system  is  chosen  to  be 
worthy  of  inclusion  but  usually  make  no  statement  regarding  the  exclusion  of  a  particular 
Service  Set  or  One  Book  Chapter  or  process/system.  One  Book  Chapter  ratings  are  built 
from  the  rated  key  processes/systems.  So,  if  no  key  processes/systems  are  identified 
under  a  particular  One  Book  Chapter,  then  that  chapter  is  not  included  in  the  RAMP  plan. 
The  same  is  true  for  Service  Set  risk  assignments,  whose  ratings  follow  those  given  to 
appropriate  One  Book  Chapters.  If  no  chapters  under  a  given  Service  Set  are  rated,  then 
that  Service  Set  is  rated  as  “NA”  or  “not  applicable”. 

So,  while  every  RAMP  plan  contains  an  Overall  rating  for  the  contractor, 
contract,  program,  or  facility  in  question,  the  plans  do  not  contain  ratings  for  every 
identified  Service  Set  and  One  Book  Chapter  formatted  into  the  RAMP  database.  Absent 
the  knowledge  of  the  decisions  made  during  the  risk  planning  phase,  consumers  of 
RAMP  data  may  not  have  a  full  understanding  of  why  individual  areas  were  not 
evaluated. 

2.  Key  Process/System  Choice 

Functional  specialists  or  process  owners  identify  key  processes/systems  from 
which  the  risk  ratings  will  be  assigned.  There  is  no  one  reference  or  master  laundry  list 
from  which  these  processes  may  be  selected.  Specialists  refer  to  a  number  of  guides, 
publications,  directives,  and  other  information  sources  specific  to  their  One  Book  Chapter 
area  to  determine  possible  process  or  system  areas  conducive  to  risk  management 
surveillance. 
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A  good  starting  place  identifying  key  processes/systems  is  the  DLAD  5000.4  One 
Book.  Each  One  Book  Chapter  provides  some  direction  regarding  the  risk  planning 
process,  but  details  and  data  references  vary  greatly  from  listing  recommended  (and 
sometimes  required)  processes  or  systems  for  surveillance  to  referencing  FAR/DFAR 
requirements  or  DoD  Directives.  Often  data  links  are  provided  to  specific  guides  and 
publications  e.g.  EVMS  Guidebook,  DSMC  Acquisition  Logistics  Guide,  DSMC  Test 
and  Evaluation  Guide,  Software  Engineering  Institute’s  Software  Capability  Maturity 
Model,  and  the  Contractor  Purchasing  System  Review  (CPSR)  Guidebook.  The  Supplier 
Risk  Management  One  Book  Chapter  3.0  is  often  referred  to  as  well. 

All  One  Book  Chapters  refer  the  specialist  to  the  applicable  contract,  appropriate 
modifications,  MOAs,  and  Letters  of  Delegation/Instruction  (LODs/LOIs)  for  their 
specific  RAMP  plans.  Given  the  contract/contractor  specific  requirements  as  well  as  the 
previously  mentioned  data  sources  for  key  process/system  identification,  the  degree  of 
variability  of  RAMP  plans  for  key  process/system  identification  is  great.  This  makes  any 
sort  of  comparative  analysis  between  the  plans  difficult.  While  a  more  systemic  “cut  and 
paste”  risk  management  plan  process  would  be  easier  to  comparatively  evaluate,  it  would 
hold  no  true  meaning  for  risk  managers.  The  plans  are  specific  to  the  contractor  and 
effort  at  hand  and  rightfully  so. 

3.  Risk  Handling  Tools 

Key  process  owners  use  a  pull-down  menu  within  the  RAMP  database  to  select 

fro  a  listing  of  risk  handling  tools  to  indicate  their  methods  of  risk  mitigation.  If  their 

tool  is  not  listed  they  may  add  their  own.  This  type  of  selection  accounts  for  the  common 

terms  used  in  most  of  the  RAMP  plans  to  indicate  risk  handling  efforts  for  specific  key 

processes/systems.  Although  many  of  these  terms  are  not  very  descriptive  in  and  of 
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themselves  (i.e.  data  analysis,  corrective  action,  product  audits,  etc.),  their  commonality 
provides  some  ability  to  comparatively  analyze  the  chosen  tools. 

“Data  Analysis”  was  the  most  often  applied  risk  handling  tool  (used  ini 084 
instances),  followed  by  “Product  Audits”  (860  uses),  “System  Evaluation”  (692  uses), 
and  “Corrective  Action”  (468  uses).  “In  process”  risk  handling  tool  assignments  were  the 
fifth  most  common  (329  uses)  and  often  associated  with  “in  process”  risk  rating 
assignments,  but  not  exclusively.  “Data  Analysis”  was  a  popular  application  across 
various  Service  Sets,  ranking  as  the  most  popular  tool  in  half  of  the  One  Book  Chapters, 
and  in  the  top  three  risk  handling  tools  for  each  of  the  chapters  80%  of  the  time. 

Information  regarding  intensity,  frequency,  and  schedule  is  a  data  entry 
requirement  for  he  tools  and  is  there  to  provide  specifics.  Some  plans  provide  verbose, 
descriptive  narratives  supporting  their  risk  handling  selection  and  application,  others 
provide  little  at  all.  Similarly,  some  plans  detail  their  risk  handling  efforts  beyond  the 
basic  risk  handling  tool  identification  within  the  narrative  (i.e.  identify  exactly  what  data 
items  undergo  “data  analysis”)  to  add  value  to  the  risk  handling  selection,  while  others  do 
not. 

So,  while  the  system  is  designed  to  provide  focal  points  (and  create  areas  of 
commonality  and  comparability)  it  simultaneously  provides  flexibility  to  match  the  risk 
handling  tool  to  the  specific  problem  or  key  process/system  identified  for  risk 
management  efforts. 

I.  CHAPTER  SUMMARY 

This  chapter  begins  with  an  introduction  of  DCMDW  and  the  sampled  RAMP 
plans  from  this  DCMA  region.  It  provides  an  overview  of  the  Overall,  Service  Set,  and 
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One  Book  Chapter  risk  ratings  for  each  of  the  42  sampled  plans.  The  Overall  risk  ratings 
and  Service  Set  summary  is  presented  and  a  brief  discussion  of  performance,  schedule, 
and  cost  follows.  Then  each  of  the  Service  Sets  is  presented  with  a  summary  and 
analysis  of  the  high,  moderate,  and  low  risk  ratings  for  each  of  the  three  areas. 
Immediately  following  each  of  the  Service  Sets  are  the  applicable  One  Book  Chapters 
and  the  chosen  key  processes/systems  for  rating  risk  for  each  of  RAMP  plans  rating  risk 
in  this  area.  A  summary  analysis  and  discussion  of  the  high  risk  areas  follows  for  each  of 
the  One  Book  Chapters.  Finally,  variations  between  the  sampled  plans  regarding  non¬ 
risk  ratings,  key  process/system  choice,  and  risk  handling  tool  methodology  are  noted. 

The  next  chapter  will  draw  conclusions  regarding  how  DCMA  addresses  risk 
management  in  the  acquisition  process  based  on  the  analysis  of  42  sampled  RAMP  plans 
within  the  DCMDW  region.  Recommendations  and  further  areas  of  recommended  study 
will  be  provided. 
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V.  CONCLUSIONS  AND  RECOMMENDATIONS 


A.  OVERVIEW 

The  focus  of  this  research  was  to  examine  how  DCMA  addresses  risk 
management  in  the  acquisition  process.  To  do  this,  the  concepts  of  risk,  risk  in 
acquisition,  and  risk  management  in  the  context  of  the  Federal  Acquisition  Process  were 
presented.  The  role  of  DCMA  in  the  post-award  contract  administration  phase  and  their 
philosophy  regarding  risk  management  was  discussed.  DCMA’s  new  information 
technology  tool  for  managing  risk,  RAMP  was  presented  and  a  sample  of  42  RAMP  risk 
management  plans  for  strategic  and  critical  suppliers  from  the  DCMDW  region  were 
studied.  Performance,  schedule,  and  cost  risk  ratings  were  examined  at  the  Overall, 
Service  Set,  One  Book  Chapter,  and  key  processes/system  levels  to  determine 
commonalities  and  consistencies  between  the  plans.  High  risk  ratings  and  their 
associated  risk  handling  tools  chosen  to  mitigate  risk  were  discussed  in  detail.  Finally, 
systemic  variabilities  and  some  conclusions  as  to  their  causes  were  presented. 

B.  CONCLUSIONS 

This  research  will  present  conclusions  by  answering  the  primary  and  subsidiary 
research  questions  proposed  in  Chapter  1 : 

1.  Subsidiary  Research  Question  1:  What  is  Risk  Management  in  the 
Context  of  the  Federal  Acquisition  Process? 

The  Federal  Acquisition  Process  assesses  and  manages  risk  through  all  three 
phases  of  pre-solicitation,  solicitation-award,  and  post-award  administration.  While  risk 
and  risk  treatments  may  vary  from  phase  to  phase,  the  five-step  DoD  risk  management 
process  remains  consistent  throughout:  risk  planning,  risk  assessment,  risk  handling,  risk 
monitoring,  and  risk  documentation. 
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Central  to  this  is  risk  identification  and  risk  analysis  as  part  of  the  risk  assessment 
phase.  Key  processes  and  systems  are  chosen  for  risk  handling  based  on  a  measure  of 
their  likelihood  of  occurrence  and  their  impact  should  the  risk  be  realized.  So  probability 
and  consequence  are  the  drivers  of  risk  assessment.  Risk  handling  can  be  treated  in  one 
of  four  ways:  risk  avoidance,  control,  transfer,  and  assumption. 

Basic  risk  management  in  the  Federal  Acquisition  Process  is  consistent  with 
methods  prescribed  elsewhere.  While  there  is  no  one  formally  directed  way  or  system  to 
manage  risk  within  Government  acquisition,  the  basic  philosophies  of  the  five-step  risk 
management  process  and  the  associated  risk  assessment  treatment  and  risk  handling 
options  remain  consistent  throughout.  The  core  foundation  is  the  same,  but  the  specifics 
and  therefore  the  ability  to  compare  risk  between  the  various  systems,  agencies,  and 
services  is  difficult  at  best.  This  is  appropriate  if  complexity  and  variation  of 
Government  and  DoD  acquisition  is  considered,  at  least  from  the  standpoint  of  deriving 
meaningful  conclusions  and  not  looking  at  wrote,  summated  descriptions  that  offer  little 
scientific  analysis  and  even  less  real  solutions  to  problem  solving  and  actual  risk 
management. 

2.  Subsidiary  Research  Question  2:  What  is  the  Defense  Contract 
Management  Agency  (DCMA)  Philosophy  with  regard  to  Risk 
Management  in  the  Post-award  Contract  Administration  Phase? 

DCMA  is  the  principal  contract  administrator  for  DoD  and  follows  the  standard 
prescribed  risk  management  process.  IPTs,  PROCAS,  and  Management  Councils  (all 
recognized  acquisition  reform  initiatives)  are  central  to  DCMA’s  risk  management  efforts 
of  using  a  comprehensive  risk  management  methodology  inclusive  of  all  stakeholders 
and  applicable  to  all  its  suppliers.  All  are  fully  incorporated  into  their  system  for 

developing  and  maintaining  risk  management  plans. 
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The  nexus  for  risk  management  within  DCMA  is  the  DLAD  5000.4  One  Book,  in 
general,  and  specifically.  Chapter  3.0  describing  the  risk  management  process  and 
assigning  responsibilities  to  all  the  CMOs.  Central  to  this  is  the  risk  matrix  structure  that 
defines  risk  in  terms  of  probability  and  consequence  and  assigns  risk  ratings  in  the  three 
areas  of  performance,  schedule,  and  cost  for  each  of  the  applicable  One  Book  Chapter 
areas  and  they’re  associated  key  processes/systems.  The  new  tool  for  managing  this  risk 
process  within  DCMA  is  RAMP,  an  information  technology  database  of  risk 
management  plans. 

The  RAMP  program  is  a  comprehensive  risk  management  tool.  It  incorporates  all 
five  phases  of  the  risk  management  process  and  allows  for  ready  reference,  quick  update, 
and  widespread  information  sharing  within  DCMA  of  risk  management  plans  for  various 
suppliers,  programs,  and  contracts.  Its  information  will  be  made  available  to  customers 
(i.e.  Program  Managers)  but  it  is  expressly  not  to  be  used  as  past  performance 
information.  While  RAMP  will  remain  closed  to  suppliers,  information  contained  within 
will  be  shared  and  discussed  through  other  mechanisms  (e.g.  IPTs  and  Management 
Councils)  to  continue  to  promote  a  teaming  and  responsive  atmosphere. 

So,  the  risk  management  philosophy  within  DCMA  is  one  of  comprehensive  and 
inclusive  management  using  information  technology  to  enhance  its  performance  and 
customer  service.  It’s  philosophy  and  techniques  are  consistence  with  the  Federal 
Acquisition  Process  and  prescribed  DoD  methodologies.  It  is  important  to  note,  that 
DCMA  sees  this  as  “risk  management”  and  not  just  “risk  handling”  or  “risk  monitoring” 
because  its  key  tool,  RAMP  incorporates  all  five  phases  of  the  risk  management  process 
and  it  is  iterative  and  timely. 
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In  actuality,  it  is  the  suppliers’  job  to  handle  the  risk  for  their  contractual  efforts. 
DCMA’s  role  is  really  one  of  assessment,  monitoring,  and  documentation  of  the 
contractor’s  systems,  processes,  and  actual  risk  handling  techniques.  But  from  an 
internal  standpoint  of  managing  suppliers,  DCMA  is  handling  risk  by  assessing  and 
monitoring  problematic  supplier  areas  (key  processes/systems)  and  supplier  methods  to 
manage  risk.  This  assessment  and  monitoring  allows  the  Government  (either  at  the 
administration  level  or  the  customer  level)  to  take  lawful  and  meaningful  contractual 
actions  to  seek  correction,  consideration,  or  resolution  that  is  fair  and  reasonable  within 
the  terms  of  Government  procurement  and  the  contract. 

3.  Subsidiary  Research  Question  3:  Are  Risk  Management  Plans  for 
Specific  Activities  Consistently  Developed  and  Applied  within 
DCMA? 

The  DLAD  5000.4  One  Book  Chapters  for  the  various  areas  addressed  in  the 

RAMP  format  for  risk  management  plans  and  indeed,  the  RAMP  program  itself  clearly 

promotes  consistent  development  and  application  of  risk  management  plans  for  the 

various  geographically  dispersed  suppliers,  contracts,  and  programs  administratively 

managed  by  DCMA.  While  the  plans  are  not  carbon  copies  of  each  other  and  there  are 

significant  variations  in  narration  style  (descriptive  and  fluid  v.  formal  and  brief),  depth 

of  justification  for  assigned  risk  ratings  (100  pages  v.  10),  and  span  of  risk  rating  areas 

(16  One  Book  Chapters  rated  v.  1)  the  plans  are  all  formatted  identically  and  use  the 

same  option  areas  for  rating  risk  down  to  the  One  Book  Chapter  level.  Key  processes 

and  system  level  risk  ratings  present  much  more  variability  given  the  large  number  of 

choices  available  to  functional  specialists/process  owners  from  generic  sources  and  the 

individual  contracts.  While  the  individual  plans  overall  may  tend  to  use  a  different  key 

process/system  combination  to  address  risk  for  an  applicable  One  Book  Chapter,  there  is 
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a  great  deal  of  overlap  between  plans  of  the  key  process/system  choices  leading  to  the 
conclusion  that  there  is  really  a  great  deal  of  commonality  in  the  choices  made,  especially 
for  the  area  that  lend  themselves  for  being  a  bit  more  systemic  and  process  oriented  (i.e. 
public  vouchers)  vice  being  more  contractually  specific  (i.e.  product  quality). 

Risk  rating  methodologies,  as  far  as  they  are  described,  appear  to  be  consistent. 
Performance,  schedule,  and  cost  are  rated  and  justified  according  to  their  probability  and 
impact  for  the  specific  key  process  or  system.  The  process/system  ratings  consistently 
flow  upward  to  justify  the  One  Book  Chapter  ratings,  Service  Set  ratings,  and  risk  ratings 
Overall.  In  some  senses,  this  is  a  “no  brainer”  because  the  RAMP  plan  promotes 
systemic  pyramiding  of  risk  ratings.  But  the  initial  key  processes  and  systems  must  be 
chosen  and  the  initial  risk  ratings  assigned  and  justified.  Narratives,  at  all  levels,  are 
subjectively  written  and  generally  support  those  rating  assignments  that  flow  upward 
from  the  process/system  assessments. 

4.  Subsidiary  Research  Question  4:  What  are  the  Areas  of  Highest  Risk 
for  Strategic  and  Critical  Suppliers  in  the  Contract  Administration 
Phase? 

Performance  was  generally  ranked  as  an  area  of  higher  risk  than  schedule  and 
cost,  although  schedule  closely  followed  due  to  their  intrinsically  close  relationship  and 
the  lowered  degree  of  control  or  affect  the  Government  can  have  over  these  areas  as 
opposed  to  cost.  From  the  Government’s  perspective  of  influence,  performance  and 
schedule  can  be  influenced  by  the  amount  of  money  the  Government  is  willing  to  pay. 
But  influence  is  not  control.  From  the  Government’s  perspective  of  function  ownership, 
performance  and  schedule  are  strictly  controlled  by  the  contractor  (i.e.,  the  Government 
isn’t  the  physical  builder  of  the  weapon  systems  it  purchases)  and  while  cost  is  incurred 
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based  upon  the  contractor’s  purchasing  and  resource  use,  it  doesn’t  necessarily  correlate 
to  price  paid  by  the  Government. 

The  Government  can  protect  itself  from  cost  overruns  through  fixed-price  type 
contracts.  That  means  the  cost  area,  at  least  from  the  perspective  of  cost  to  the 
Government,  is  controllable  by  the  Government  to  a  higher  degree  than  performance  and 
schedule  which  ultimately  lies  in  the  hands  of  the  contractor.  Of  course  this  assumes  the 
contractor  will  eventually  delivery  the  purchased  performance  and  no  additional  costs 
will  be  incurred  by  the  Government  due  to  delay  or  failure.  Whether  this  risk  is 
appropriately  accounted  for  under  the  cost  area  for  a  specific  contractor,  contract,  or 
program  in  question  is  debatable.  Obviously  risk  in  one  area  will  drive  risk  in  the  other 
areas.  But  if  this  is  carried  too  far,  then  dividing  risk  into  the  three  categories  of 
performance,  schedule,  and  cost  makes  no  sense  and  one  risk  rating  will  suffice.  From 
this  view,  it  is  reasonable  to  conclude  some  isolation  of  these  three  areas  is  appropriate 
otherwise  they  are  meaningless.  Generally,  risk  associated  with  failure  to  perform  or 
schedule  was  addressed  under  the  performance  and  schedule  area  and  cost  was  treated 
separately. 

Product  Support  was  the  area  of  highest  risk  among  the  Service  Sets.  It  was  the 
most  applied  Service  Set,  over  83%  of  the  42  plans  rated  risk  in  this  area.  This  is  due  in 
part  to  the  numerous  One  Book  Chapters  associated  with  this  area,  more  than  twice  as 
many  as  any  other  area.  Supply  QA  -  Product  Quality  was  the  riskiest  One  Book 
Chapter  in  the  RAMP  system  for  the  sample;  over  76%  of  the  plans  assessed  risk  in  this 
area.  Product  Quality  is  a  One  Book  Chapter  within  the  Product  Support  Service  Set  and 
further  supports  the  “most  risky”  designation.  Just  as  Product  Support  contains  the 
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majority  of  One  Book  Chapter  risk  areas  to  support  increase  its  likelihood  of  risk 
management  efforts,  Product  Quality  included  far  more  identified  key  processes/systems 
than  any  other  One  Book  Chapter;  140  different  processes/systems  were  identified  from 
the  42  sampled  plans. 

Apart  from  the  sheer  numbers  aspects  of  available  options  to  rationalize  the  high 
risk  associated  with  Product  Support  in  general  and  Product  Quality  specifically,  this  area 
intrinsically  lends  itself  to  being  the  most  risky.  Of  all  the  areas  rated,  it  seems  to  be  the 
most  exactly  tied  to  the  specifics  of  contract  performance  over  more  generic  and  systemic 
risk  areas.  They  are  in  congruence  with  the  high  risk  conclusions  regarding  the  areas  of 
performance  and  schedule  previously  mentioned.  Delivery  is  the  second  most  frequently 
applied  Service  Set  (over  76%)  and  Schedule  and  Delivery  Management  the  second  most 
frequently  applied  One  Book  Chapter  risk  (over  69%). 

5.  Subsidiary  Research  Question  5:  What  are  the  Most  Common  Tools 
used  to  Mitigate  Risk  in  Key  Processes  and  Systems? 

“Data  Analysis”,  “Product  Audits”,  “System  Evaluation”,  and  “Corrective 
Action”  were  the  most  commonly  applied  risk  handling  tools  in  the  sampled  RAMP 
plans.  While  these  tools  are  common  enough  for  a  comparative  analysis  across 
individual  RAMP  plans.  Service  Sets,  One  Book  Chapters,  and  processes/systems  they 
are  too  broad  for  great  detail  or  depth  of  analysis.  When  the  assigned  tools  are  supported 
by  accurate  intensity,  frequency,  and  schedule  information  as  well  as  narratives 
describing  the  selection  of  tools,  and  their  applicability,  they  become  both  meaningful  and 
comparable.  The  quality  of  the  narratives,  their  descriptiveness  and  ability  to  provide 
rationale  to  support  the  tool  selection  are  paramount  to  RAMP’s  usefulness.  Without  it, 
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they  remain  comparable  at  a  macro  level,  but  without  depth  and  substance  required  for 
detailed  use  and  applicability. 

The  common  risk  handling  tools  highlight  DCMA’s  role  to  analyze,  monitor,  and 
survey  and  prompt  the  contractor  when  necessary.  These  techniques  allow  the 
Government  to  assess  problems  and  work  with  the  contractor  to  fix  them  using  risk 
handling  techniques  the  contractor  must  ultimately  employ.  They  focus  on  actions  before 
the  final  inspection,  on  procedural  and  systemic  problems  to  achieve  real  change  in  fixing 
the  root  cause  of  production  difficulty.  This  allows  DCMA  to  prioritize  process 
improvement  opportunities  and  allocate  resources  from  a  risk-based  perspective. 

6.  Primary  Research  Question:  How  Does  the  Defense  Contract 
Management  Agency  (DCMA)  Address  Risk  Management  in  the 
Acquisition  Process? 

DCMA  uses  a  comprehensive,  inclusive,  and  iterative  approach  to  risk 
management.  It  follows  the  Government  and  DoD  risk  management  premise  of  using  a 
five-step  approach  to  risk  management  and  the  basic  idea  of  identifying  and  assessing 
key  processes/systems  whose  risk,  either  through  probability  or  potential  impact,  offers 
the  most  cause  for  concern  from  a  performance,  schedule,  or  cost  perspective.  It  employs 
current  information  technology,  RAMP  to  provide  consistency,  commonality,  access  and 
comparability  to  its  risk  management  process. 

DCMA  and  the  RAMP  process  for  risk  management  naturally  focus  on  and 
explore  high  risk  areas  given  the  nature  of  identifying  and  establishing  key 
processes/systems  and  the  requirement  for  written  narratives  at  every  level  of  assessment. 
The  high  risk  areas  tend  to  be  related  to  performance  and  schedule,  product  support  and 
product  quality,  and  delivery.  But  risk  management  in  the  post-award  phase  requires 
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DCMA’s  risk  management  to  be  more  akin  to  risk  assessment  and  monitoring  than 
actually  handling  the  risk,  because  it’s  really  the  supplier  who  has  the  direct  ability  to 
make  change  and  handle  risk  associated  with  his  processes  and  systems.  However,  risk 
handling  does  occur  from  indirect  means  provided  through  risk  assessment,  risk 
monitoring,  and  contractually  corrective  actions  consistent  with  procurement  laws  and 
the  terms  of  the  contract.  The  contractor’s  non-responsiveness  is  determined  by  risk 
management  which  is  a  focus  on  those  things  that  are  important  vice  checking 
everything.  From  a  cost-benefit  approach,  this  makes  economic  sense. 

C.  RECOMMENDATIONS 

1.  DoD  Should  Mandate  a  Common  Risk  Management  Process 
throughout  all  DoD  Organizations  and  Applicable  to  Each  of  the 
Services,  Agencies,  and  Acquisition  Offices 

In  times  of  tightening  defense  budgets  and  fewer  manpower  resources,  DoD  must 
find  more  cost  efficient  ways  to  ensure  quality  is  delivered  by  its  contractors.  It  can  no 
longer  depend  on  100%  final  inspection  as  its  primary  means  of  surveillance.  By 
evaluating  high  risk  areas,  based  on  probability  and  impact,  DoD  acquisition 
organizations  can  focus  their  attention  on  the  areas  where  they  are  likely  to  reap  the  most 
results  from  a  perspective  of  cost,  time,  and  manpower  input — basically  a  form  of  cost- 
benefit  analysis. 

The  risk  management  process  as  defined  in  the  DSMC  Risk  Management  Guide  is 

good  starting  point.  However,  there  are  numerous  differing  and  specific  plans  employed 

by  each  of  the  services,  agencies,  and  individual  commands.  Achieving  a  common 

approach  is  best  achieved  by  mandating  a  common  information  technology  tool.  This 

would  forward  the  ideas  of  interoperability  between  the  services  and  the  application  of 

consistent  Government  acquisition  practices.  Having  a  common  methodology  that  is 
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directly  comparable  amongst  all  DoD  organizations  improves  the  Government’s  ability  to 
learn  from  itself,  make  better  acquisition  decisions,  and  present  one  face  to  contractors. 
Commonality  would  open  individual  acquisitions  to  more  competitors  who  are  able  to 
better  understand  the  common  DoD  approach  and  therefore  better  able  to  compete. 

As  it  is  currently  written,  RAMP  would  not  address  necessary  risk  management 
areas  prevalent  in  other  phases  of  the  acquisition  cycle  and  it  references  DCMA  specific 
directives  that  are  not  applicable  elsewhere.  So,  RAMP  is  not  suitable  for  these  purposes. 
Certainly,  any  information  technology  solution  would  have  to  be  greatly  expanded  and 
very  flexible  to  accommodate  the  differing  needs  and  requirements  of  the  various 
acquisition  activities  and  services.  But  if  a  common  acquisition  system  across  DoD  is  an 
acquisition  reform  goal  then  moving  in  that  direction  makes  formulating  such  a  program 
much  more  feasible  and  cost  effective  because  the  variances  will  be  fewer.  Of  course, 
from  the  direct  opposite  position,  by  formulating  a  DoD-wide  risk  management  plan  and 
data  system  the  acquisition  arena  would  move  closer  to  having  a  common  system. 
Designing  the  process  and  mandating  the  specifics  is  the  first  step.  The  information 
technology  tool  should  follow. 

2.  Revise  the  RAMP  Plan  Format  to  Make  them  Even  More  Directly 
Comparable  to  Each  Other  and  Incorporate  a  Summated  Spreadsheet 
Linked  to  the  Risk  Ratings  for  Each  Area 

Currently  all  Service  Sets  are  presented  even  when  not  rated:  “not  applicable” 

however,  only  those  One  Book  Chapters  and  key  processes/systems  actually  rated  are 

presented  in  the  final  risk  management  plan.  Expand  the  formatting  to  include  all  One 

Book  Chapters  for  all  rated  Service  Sets  in  the  final  RAMP  plan  whether  or  not  they  are 

rated  to  allow  for  a  more  direct  comparison  of  Service  Sets  between  plans.  Additionally, 

for  each  of  the  One  Book  Chapters  that  are  rated  prescribe  some  basic  key 

112 


processes/system  to  be  commonly  listed  on  all  RAMP  plans  whether  they  are  rated  or 
not.  Any  additions  to  these  more  common  areas  could  be  added  beneath  to  provide  for 
needed  detail  and  individuality  of  contractual  efforts.  These  additions  along  with  written 
narratives  justifying  risk  ratings  provide  the  necessary  flexibility  to  include  meaningful 
data  for  specific  risk  management  plans  into  a  common  and  generic  system  that  is 
flexible  enough  to  allow  for  deviations. 

Incorporate  a  summated  spreadsheet,  similar  to  that  presented  in  Appendix  B, 
linked  to  the  actual  risk  ratings  for  each  of  the  plans  at  least  to  the  One  Book  Chapter 
level.  This  would  allow  for  quick  and  easy  direct  comparison  between  plans  and  provide 
an  accurate  means  to  perform  statistical  analysis  and  draw  summary  conclusions  about 
how  risk  is  managed  within  a  given  office  or  geographic  region. 

3.  Use  RAMP  Data  for  Past  Performance  Information 

The  information  currently  populating  the  RAMP  database  does  not  fit  the  DoD 

definition  of  Past  Performance  Information  (PPI).  This  seems  like  an  incredible  waste  of 
time  and  talent  and  information.  The  data  within  RAMP  are  factual  and  current.  It 
studies  the  contractor’s  processes  and  systems  in  terms  of  performance,  schedule,  and 
cost,  all  relevant  PPI  issues.  Either  these  data  and  the  way  they  are  collected  and 
presented  should  be  modified  to  conform  to  DoD  requirements  for  PPI  or  the  DoD 
requirements  regarding  the  collection  and  use  of  PPI  should  be  changed  to  accommodate 
the  wealth  of  information  gathered  in  RAMP. 

High  risk  ratings  for  suppliers  may  be  a  touchy  issue  from  their  perspective.  But 
it  is  important  to  remember  that  this  rating  reflects  risk  and  not  necessarily  performance. 
Because  the  focus  of  RAMP  data  is  to  handle- risky  areas,  it  may  not  and  probably  does 
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not  address  solid  performance  areas  for  contractors.  Therefore,  RAMP,  if  used  as  PPI 
should  only  be  one  of  several  PPI  sources  because  its  information  is,  by  definition, 
limited  to  problem  areas  and  not  performance  as  a  whole.  But  if  used  as  one  piece  of  the 
past  performance  map  for  a  contractor,  it  will  provide  good  detail  regarding  how 
responsive  a  contractor  is  once  risk  is  identified,  how  able  a  contractor  is  in  identifying 
and  handling  their  risk,  and  their  propensity  to  work  with  the  Government  to  resolve 
problem  issues.  Contractors  would  need  to  be  granted  access  to  RAMP  data  if  used  for 
PPI  which  is  not  a  far  step  from  the  Government’s  desire  to  share  data  with  the  contractor 
prior  to  data  upload. 

D.  SUGGESTED  AREAS  FOR  FURTHER  RESEARCH 

Suggested  topics  for  further  research  include: 

1.  Identify  and  Compare  Various  Risk  Management  Models  and  IT 
Systems  in  Use  in  DoD 

What  are  the  areas  of  convergence  and  divergence  between  the  models?  How  can 
risk  management  systems  be  modified  to  provide  a  “fit”  for  all  the  Services,  agencies, 
and  acquisition  offices  in  DoD  either  in  one  IT  system  or  in  multiple,  highly 
interoperable  systems? 

2.  Study  the  RAMP  Program  from  an  IT  and  Process  Oriented 
Perspective 

Does  RAMP  improve  or  enhance  the  risk  management  process  at  DCMA?  Is  the 
RAMP  program  “user  friendly”?  Are  DCMA  personnel  adequately  trained  to  use 
RAMP?  Does  the  automated  RAMP  process  provide  meaningful  data  to  users?  Is  the 
RAMP  process  itself  a  faster  and  more  efficient  means  of  creating,  updating,  and  using 
risk  management  plans?  How  can  the  RAMP  processes  for  data  input,  modification, 
retrieval,  and  dissemination  be  improved? 
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3.  Research  Whether  RAMP  and  Other  Risk  Management  Activities  at 
DCMA  Actually  Reduce  Acquisition  Risk 

Overtime,  do  the  risk  ratings  improve?  This  can  be  studied  from  the  standpoint  of 
individual  RAMP  plans  overtime  actually  documenting  risk  reduction,  reduced  risk  from 
the  perspective  of  all  RAMP  plans  written  by  specific  geographic  or  in-plant  offices,  or 
risk  reductions  documented  for  the  regions  or  DCMA  activities  as  a  whole.  Risk 
reductions  can  be  viewed  from  the  standpoint  of  fewer  area  being  included  and  monitored 
within  RAMP  and  from  the  standpoint  of  high  risk  ratings  reduced  to  moderate  or  low 
ratings. 
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APPENDIX  A.  RAMP  INVENTORY 


INDEX  OF  SAMPLED  RAMP  PLANS 

PLAN#  DCMDW  OFFICE/CONTRACTOR  PROGRAM/CONTRACT 


DCM  RAYTHEON,  LOS  ANGELES 

1  RAYTHEON  CO  C3S 

2  RAYTHEON  ELEC  SYS  (SAT) 

3  RAYTHEON  ELEC  SYS  (SAT) 

4  RAYTHEON  ELEC  SYS  (SAT) 

5  RAYTHEON  ELEC  SYS  (SAT) 

6  RAYTHEON  ELEC  SYS  (SAT) 

DCM  SAN  ANTONIO 

7  HUNTSVILLE  AVIATION 

8  BOEING  AEROSPACE  SUPT  CTR 

9  RAYTHEON  AIR 

1 0  BROWN  &  ROOT  SVCS  CORPS 

11  EXXON/MOBIL 

12  LOCKHEED  MARTIN  HARLINGEN 

13  GRACOIND 

14  WESTINGHOUSE  ELEC  CORP 

15  SOUTHWEST  AIRPORT  SVCS 

16  DYNA-MARQ 

17  STEWART  &  STEVENSON 

18  D&D  MACHINERY  &  SALES  INC 

19  DAVIES  RAIL  &  MECHANICAL 

DCM  PHOENIX 

20  MOTOROLA  SSG 

21  MOTOROLA  SSG 

22  MCDONNEL  DOUGLAS  HELICOPTER  SYS 

DCM  DALLAS 

23  LOCKHEED  MARTIN  MISSILES  &  FC 

24  LOCKHEED  MARTIN  MISSILES  &  FC 

25  DRS  INFRARED  TECHNOLOGIES  LP 

26  DRS  INFRARED  TECHNOLOGIES  LPD 

27  AMORPHOUS  MATERIALS 

DCM  VAN  NUYS 

28  LITTON  SYSTEM  INC,  G&C  SYS  DIV 

29  SAMS  AIRPACK  PLUS  INC 

30  AEROJET  GENERAL  CORP 

DCM  SANTA  ANA 

31  PARKER  HANNIFIN  CUSTOMER 

32  APPLIED  MATERIAL  TECH 

33  AEROJET 

34  HONEYWELL  ENGINES  &  SYSTEMS 

DCM  LOCKHEED  MARTIN,  SUNNYVALE 

35  UNITED  TECH  CORP 

36  ASSOC  AEROSPACE  ACT 

37  TELECHEM  INTL  INC 

38  NORTHROP  GRUMMAN 

DCM  RAYTHEON,  TUCSON 

39  RAYTHEON  TUCSON 

40  RAYTHEON  TUCSON 

41  RAYTHEON  TUCSON 

42  RAYTHEON  TUCSON 


EPLRS  (PRIME) 

F/A 18  E/R  HORNET  (SUPPORT) 
F 15  (SUPPORT) 

F 15  (PRIME) 

F 18  SPARE/SUPPORT  (PRIME) 
TPCM7FIREFINDER  (PRIME) 


FACILITY 

KC-10CLS  (PRIME) 

FACILITY 

FACILITY 

SPO600 ...  (PRIME) 

EELV  (SUPPORT),  TITAN  IV  (FACILITY) 
C&T  (PRIME) 

FACILITY 

FACILITY 

FACILITY 

FACILITY 

FACILITY 

FACILITY 


F-22  (SUPPORT),  F-22  (FACILITY) 
MAVSTAR  GPS  (SUPPORT) 
LONGBOW  APACHE  (PRIME) 


ATACMS-BAT  (PRIME)  (FACILITY) 

HIMARS  (PRIME) 

Ml A2  ABRAMS  UPGRADE  (PRIME)  (FACILITY) 
JAVELIN  DDC  SUBCONTRACT  (SUPPORT) 
AV-8B  REMANUFACTURE  (PRIME) 


FACILITY 

FACILITY 

TITAN  IV  (FACILITY) 


FACILITY 

FACILITY 

SADARM  (PRIME)  (FACILITY) 

DOD  (SUPPORT),  F-22  (FACILITY) 


MINUTEMAN  III  PRP  (FACILITY) 

FACILITY 

FACILITY 

TRIDENT  II  MISSILE  (PRIME)  (FACILITY) 


TOMDEP  (PRIME)  TOMAHAWK  (FACILITY) 
AMRAAM  (PRIME) 

SYSTEMS  (PRIME) 

ESSM  (PRIME) 
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Supplier  Quality  Assurance  -  Quality  System 
Supplier  Quality  Assurance  -  Product  Quality 
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